-- Logs begin at Tue 2022-08-23 21:25:32 UTC, end at Tue 2022-08-23 22:44:28 UTC. -- Aug 23 22:07:37.095178 dv-cinder-850321-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 23 22:07:37.110372 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 23 22:07:37.112943 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 23 22:07:37 2022] *** Aug 23 22:07:37.112943 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 23 22:07:37.112943 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 23 22:07:37.112943 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: nodename: dv-cinder-850321-4-iscsi Aug 23 22:07:37.112943 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: machine: x86_64 Aug 23 22:07:37.112943 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: clock source: unix Aug 23 22:07:37.112943 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: pcre jit disabled Aug 23 22:07:37.112943 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: detected number of CPU cores: 4 Aug 23 22:07:37.112943 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: current working directory: / Aug 23 22:07:37.112943 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: detected binary path: /usr/bin/uwsgi-core Aug 23 22:07:37.112943 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: your processes number limit is 47680 Aug 23 22:07:37.112943 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: your memory page size is 4096 bytes Aug 23 22:07:37.114115 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: detected max file descriptor number: 2048 Aug 23 22:07:37.114115 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: lock engine: pthread robust mutexes Aug 23 22:07:37.114115 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: thunder lock: enabled Aug 23 22:07:37.114115 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 23 22:07:37.114115 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 23 22:07:37.153335 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: Python main interpreter initialized at 0x55c120a2c9c0 Aug 23 22:07:37.153335 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: python threads support enabled Aug 23 22:07:37.153335 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: your server socket listen backlog is limited to 100 connections Aug 23 22:07:37.153335 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: your mercy for graceful operations on workers is 90 seconds Aug 23 22:07:37.153833 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: mapped 403077 bytes (393 KB) for 2 cores Aug 23 22:07:37.153833 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: *** Operational MODE: preforking *** Aug 23 22:07:37.154085 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: *** uWSGI is running in multiple interpreter mode *** Aug 23 22:07:37.154085 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: spawned uWSGI master process (pid: 81670) Aug 23 22:07:37.154407 dv-cinder-850321-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 23 22:07:37.157771 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: spawned uWSGI worker 1 (pid: 81679, cores: 1) Aug 23 22:07:37.157771 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: spawned uWSGI worker 2 (pid: 81680, cores: 1) Aug 23 22:07:37.157771 dv-cinder-850321-4-iscsi devstack@keystone.service[81670]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 23 22:07:42.038166 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 22:07:42.038166 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: from cryptography.utils import int_from_bytes Aug 23 22:07:42.039388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 22:07:42.039388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: from cryptography.utils import int_from_bytes Aug 23 22:07:42.044545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 22:07:42.044545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: from cryptography.utils import int_from_bytes Aug 23 22:07:42.045421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 22:07:42.045421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: from cryptography.utils import int_from_bytes Aug 23 22:07:42.094775 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 23 22:07:42.095024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 23 22:07:42.095174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 23 22:07:42.095336 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] command line args: [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 23 22:07:42.095469 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 23 22:07:42.095603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] command line args: [] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 23 22:07:42.095744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 23 22:07:42.095884 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ================================================================================ {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 23 22:07:42.096014 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 23 22:07:42.096183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] admin_token = **** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.096327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ================================================================================ {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 23 22:07:42.096543 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] config_dir = [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.096699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] admin_token = **** {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.096892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.097084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] config_source = [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.097221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] config_dir = [] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.097366 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] debug = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.097509 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.097650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.101041 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] config_source = [] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.101041 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] debug = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.101503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.101503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.102601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.102827 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.102827 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.103191 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.103390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] list_limit = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.103568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.103764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] log_config_append = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.103764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.103764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] log_dir = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.104274 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.104473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] log_file = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.104660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.104853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.105035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] list_limit = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.105251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.105423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] log_config_append = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.105635 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.105828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.106012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.106202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] log_dir = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.106394 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.106582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] log_file = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.106582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.106935 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.107121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.107312 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.107496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.107679 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.107883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.108079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.108270 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.108462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.108642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.108834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.109021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.109251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.109446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.109641 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.109641 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.109997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.110187 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.110385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.110385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.110751 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.110751 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] publish_errors = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.111114 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.111293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.111478 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.111668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.111668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.112035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.112222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.112222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.112560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.112744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] standard_threads = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.112744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.112744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.113347 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.113573 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.113764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] publish_errors = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.113949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] use_journal = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.113949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] use_json = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.114307 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.114495 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] use_stderr = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.114495 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] use_syslog = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.114911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.115093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.115275 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.115457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.115651 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.115846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.115846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.116189 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.116380 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.116561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] standard_threads = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.116760 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.116760 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.117125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.118340 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.118528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.118723 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.118723 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.119318 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.119515 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.119710 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] use_journal = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.119903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.120085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] use_json = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.120289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.120472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] use_stderr = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.120653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.120837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] use_syslog = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.121018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.121270 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:07:42.121466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.121466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.121806 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.121987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.122170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.122362 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.122362 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.122714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.122893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.123076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.123254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.123254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.123608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.123789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.123789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.124131 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.124311 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.124494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.124796 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.124796 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.125183 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.125367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.125367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.125729 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.125926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.126105 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.126289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.126289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.126637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.126827 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.127019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.127216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.127216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.127564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.127745 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.127933 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.128128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.128128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.128467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.128650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.128830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.129022 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.129022 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.129462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.129646 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.129828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.130021 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.130021 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.130391 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.130574 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.130759 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.130949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.130949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.131303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.131486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.131486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.131828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.132011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.132191 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.132387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.132387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.132733 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.134278 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.134473 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.134677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.134677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.135147 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.135330 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.135514 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.135709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.135709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.136068 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.136251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.136435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.136618 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.136618 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.136966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.137250 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.137435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.137628 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.137628 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.137998 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.138183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.138388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.138553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.138553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.138863 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.139053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.139220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.139397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.139563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.139726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.139726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.140045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.140264 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.140454 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.140620 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.140620 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.140943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.141148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.141148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.141527 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.141725 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.141919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.142106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.142106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.142453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.142652 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.142852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.143038 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.143038 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.143400 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.143584 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.143769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.143971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.143971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.144327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.144512 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.144696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.144889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.144889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.146697 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.146880 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.147055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.147228 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.147399 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.147588 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.147978 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.148170 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.148170 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.148562 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.148736 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.148935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.149164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.149332 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.149526 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.149713 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.149883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.150066 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.150249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.150499 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.150675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.150879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.151071 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.151273 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.151488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.151488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.151846 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.152014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.152183 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.152368 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.152548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.152763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] auth.external = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.152980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.153195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.153390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.153578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.153751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.153919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.154101 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.154266 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] auth.password = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.154470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.154874 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] auth.token = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.155072 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.155242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.155242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.155242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.155751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.155948 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.155948 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.156340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.156526 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.156526 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.156883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.157124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] credential.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.157328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.157535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.157535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.157872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.158058 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.158247 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.158452 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.158452 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.158808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.159008 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.159170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.159347 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.159347 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.159682 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.159850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.160033 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.160210 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.160403 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.160606 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.160606 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.160968 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.160968 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.161341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.161341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.161753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.161946 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.162133 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.162333 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] federation.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.162503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.162674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.162674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.162986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.163172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.163357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.163540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.163738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.163923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.164191 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.164393 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.164580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.164765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.164955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.165196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.165196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.165635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.165823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.165823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.165823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.166346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.166530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.166731 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.166916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.166916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.167261 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.167448 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.167634 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.167836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.167836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.168183 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.168369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.168567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.168767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.168969 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.169191 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.169191 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.169540 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.169724 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.169724 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.170084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.170268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.170453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.170636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.170636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.170981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.171177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.171364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.171549 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.171549 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.171896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.172088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.172289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] auth.external = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.172476 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.172783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.172928 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.172928 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.173252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.173409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.173409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.173652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.173782 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.173915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] auth.password = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.174047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.174047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.174303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] auth.token = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.174435 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.174567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.174699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.174699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.174956 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.175085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.175217 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.175350 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.175350 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.175601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.175851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.175984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.176116 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.176116 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.176356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.176500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.176639 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.176772 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.176902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] credential.caching = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.177046 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.177342 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.177495 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.177495 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.177737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.177934 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.178070 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.178202 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.178352 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.178483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.178613 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.178740 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.178740 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.178979 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.179128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.179264 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.179397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.179527 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.179656 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.180375 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.180520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.180520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.180762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.180893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.181025 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.181238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.181393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.181508 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.181624 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.181814 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.181946 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.182078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.182224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.182354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.182354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.182597 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] federation.caching = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.182728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.182860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.183006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.183138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.183269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.183403 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.183534 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.183665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.183810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.183943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.184072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.184207 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.184340 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.184552 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.184716 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.184939 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.185144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.185324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.185462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.185682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.185839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.186045 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.186248 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.186390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.186650 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity.caching = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.186793 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.187010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.187150 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.187358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.187495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.187722 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.187891 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.188100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.188318 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.188460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.188750 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.188928 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.189118 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.189284 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.189469 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.189648 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.189838 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.190064 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.190215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.190465 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.190611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.190845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.190997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.191190 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.191331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.191553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.191697 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.191917 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.192058 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.192280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.192454 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.192648 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.192876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] resource.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.193021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.193553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.193699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.193968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.194123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.194318 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.194460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.194665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.194802 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.195016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.195224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.195421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.196220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.196590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.196731 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.196879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.197011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.197279 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.197499 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.197707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.197916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.198133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] role.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.198337 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.198552 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] role.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.198758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.198959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.198959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.199368 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.199571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.199777 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.199981 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.199981 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.200360 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.200572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.200772 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.200974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.201265 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.201469 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.201469 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.201861 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.202080 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.202209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.202333 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.202333 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.202580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.202727 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.202727 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.202975 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.203106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.203291 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.203512 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.203669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.203931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.204076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.204298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.204437 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.204658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.204814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.205024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.205410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.205599 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.205823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.206010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.206180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.206423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.206567 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.206823 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.206966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.207163 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.207300 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.207507 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.207645 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.207974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.208132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.208358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.208499 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.208714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.208858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.209082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.209243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.209462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.209654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.209796 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.210059 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.210213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] token.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.210461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.210595 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.210730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.210899 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.211109 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.211344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.211503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.211748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.211891 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.212101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.212241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.212461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.212615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.212817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.212957 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.213202 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.213392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.213551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.213813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.213956 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.214165 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.214303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.214526 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.214665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.214894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.215034 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.215243 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.215445 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.215589 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.215842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.216001 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.216245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.216388 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.connection = **** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.216650 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.216807 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.217021 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.217208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.217423 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.217582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.217789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.217929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.218973 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.219565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.219703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.219703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.219963 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.220096 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.220231 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.220366 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.220366 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.220606 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.220739 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.220870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.221002 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.221002 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.221285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.221431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.221564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.221697 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.221827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.221962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.222107 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.222238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.222370 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.222502 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.222633 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.222765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 23 22:07:42.223044 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] resource.caching = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.223398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.223748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.224086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.224438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.224788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.225177 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.225527 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.225863 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.226200 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.226535 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] role.caching = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.227460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] role.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.227799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.228149 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.228531 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.228881 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.229272 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.229622 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.229981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.230330 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.230680 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.231014 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.231363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.231712 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.232067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.232414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.232949 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.233316 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.233669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.234056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.234414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.234765 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.235099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.235452 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.235803 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.236138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.236476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.236806 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.237188 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.237539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.237891 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.238235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.238612 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.238946 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.239276 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.239609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] token.caching = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.239945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.240290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.240657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.241034 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.241429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.241790 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.242154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.243172 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.243556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.243918 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.244279 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.244638 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.245008 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.245419 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.245780 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.246153 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.246571 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.246955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.connection = **** {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.247334 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.247713 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.248074 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.248446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.248805 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.249202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.249561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.249920 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.250287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.250653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.251018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.251376 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.251745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.252103 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.252472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.252825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.253224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.253580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:07:42.253859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81679) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 23 22:07:42.267478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.267858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.268247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.281563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.281912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.282227 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.285404 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.292217 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.323705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.323961 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.333158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.333428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.395113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.396871 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.451323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.451589 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.451764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.451922 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.531414 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.531755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.532111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.532461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.532806 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.533197 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.575510 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.575772 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.575944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.576227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.576454 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.576591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.576801 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.589844 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.628537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.628925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.629365 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.629776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.630167 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.630575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.653518 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.653770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.653953 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.654176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.654397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.654618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.654837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.663939 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.677968 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.678196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.678405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.678627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.678845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.679061 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.679277 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81679) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.697801 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.698067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 22:07:42.698362 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.698563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 22:07:42.698841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.699108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.699682 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.699951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.703720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.704069 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.704383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.704691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.705004 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.705335 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.705635 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:07:42.709646 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.709947 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.710225 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.710517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.717676 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.717986 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 22:07:42.718429 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.718714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 22:07:42.718898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.719147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.719504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.719854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.720197 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.722031 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.722333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.722788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.722940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.723191 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.723458 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.723815 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.724058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.724470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.724751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.725027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.732281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.732675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.733031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.733420 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.740766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.744457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.744839 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.745250 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.745621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.745990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.746339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.746705 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.747068 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.747426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.747808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.748164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.749406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.750922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.751211 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.756030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.756326 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.756618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.764078 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.764381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.768576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.768875 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.769189 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.769480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.769760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.770040 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.770319 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.770592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.776471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.782129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.782129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.783113 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.783461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.783835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.788046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.788339 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.790779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.791141 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.792462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.796168 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.796487 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.796919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.797463 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.797942 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.798430 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.798918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.800358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.800358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.800358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.800857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.804070 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.807940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.808245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.808545 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.808841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.809164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.809453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.817872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.818242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.823661 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.826763 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.826995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.827147 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.827287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.827459 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.827598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.831857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.834849 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.835131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.837032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.837338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.838826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.839134 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.839421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.839843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.840075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.840283 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.840428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.840567 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.840703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.840951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.841220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.853379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.853673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.853961 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.854246 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.854533 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.854813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.857401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.857777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.858105 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.858429 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.865321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.868494 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.868859 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.870866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.871708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.872092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.872443 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.872856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.873239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.874117 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.877015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.878490 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.880066 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.880353 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.880652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.885137 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.885497 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.885837 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.886202 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.886556 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.886905 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.887251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.887402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.888733 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.891780 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.892097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.892396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.892683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.901374 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.901677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.901969 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.904233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.907254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.909106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81679) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.909379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.909714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.909998 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.910270 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.910546 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.910834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.910983 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.911124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.911394 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.911674 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.911950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.912214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.912497 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.912740 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.912890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81679) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.915062 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.915420 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.915784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.922439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.924012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.927092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.927449 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.927795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.928133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.936981 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.937373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.937727 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.945025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:07:42.946150 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.946509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.946858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.947354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.947780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.948189 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.948542 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.948901 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.949281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.949623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.949829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c120a2c9c0 pid: 81679 (default app) Aug 23 22:07:42.950010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.950357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.950695 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:07:42.953647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f07bd6-be4a-45b0-93ad-0408f69c3b77 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:42.953934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f07bd6-be4a-45b0-93ad-0408f69c3b77 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:42.954208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f07bd6-be4a-45b0-93ad-0408f69c3b77 None None] PATH_INFO: `/v3` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:42.958507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1/1] 10.222.0.88 () {50 vars in 791 bytes} [Tue Aug 23 22:07:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:07:42.990200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c120a2c9c0 pid: 81680 (default app) Aug 23 22:07:50.411896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38f483-0ed7-417f-9590-63762c2d9177 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:50.412526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38f483-0ed7-417f-9590-63762c2d9177 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:50.412788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38f483-0ed7-417f-9590-63762c2d9177 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:50.415233 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 2/2] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:07:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:07:50.428291 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae99a1-7df1-4b83-8838-67d3298703cb None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:50.428901 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae99a1-7df1-4b83-8838-67d3298703cb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:50.429871 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae99a1-7df1-4b83-8838-67d3298703cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:50.587090 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: INFO dbcounter [None req-2dae99a1-7df1-4b83-8838-67d3298703cb None None] Registered counter for database keystone Aug 23 22:07:50.622666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG oslo_db.sqlalchemy.engines [None req-2dae99a1-7df1-4b83-8838-67d3298703cb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81680) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 23 22:07:50.634282 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writer thread running {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 23 22:07:50.908149 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG passlib.handlers.bcrypt [None req-2dae99a1-7df1-4b83-8838-67d3298703cb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81680) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 23 22:07:50.908864 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG passlib.handlers.bcrypt [None req-2dae99a1-7df1-4b83-8838-67d3298703cb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81680) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 23 22:07:50.972523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2dae99a1-7df1-4b83-8838-67d3298703cb None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:07:50.974519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2dae99a1-7df1-4b83-8838-67d3298703cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:07:51.145610 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1/3] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:07:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 724 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 23 22:07:51.638853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38f483-0ed7-417f-9590-63762c2d9177 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:07:51.786561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: INFO dbcounter [None req-1d38f483-0ed7-417f-9590-63762c2d9177 None None] Registered counter for database keystone Aug 23 22:07:51.820770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG oslo_db.sqlalchemy.engines [None req-1d38f483-0ed7-417f-9590-63762c2d9177 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81679) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 23 22:07:51.830482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writer thread running {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 23 22:07:52.066496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38f483-0ed7-417f-9590-63762c2d9177 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:07:52.075047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec46919-9c72-49d7-ac0c-bf32226fa8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:07:52.076133 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec46919-9c72-49d7-ac0c-bf32226fa8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:52.076605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec46919-9c72-49d7-ac0c-bf32226fa8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:52.077026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec46919-9c72-49d7-ac0c-bf32226fa8e6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:52.093495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec46919-9c72-49d7-ac0c-bf32226fa8e6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:07:52.190883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cec46919-9c72-49d7-ac0c-bf32226fa8e6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 23 22:07:52.191224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec46919-9c72-49d7-ac0c-bf32226fa8e6 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:07:52.203162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 3/4] 10.222.0.88 () {58 vars in 1324 bytes} [Tue Aug 23 22:07:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:07:52.209370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae99a1-7df1-4b83-8838-67d3298703cb None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:07:52.226026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae99a1-7df1-4b83-8838-67d3298703cb None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:07:52.231101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545251f6-8621-4b8d-adc9-139048789203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:07:52.232275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545251f6-8621-4b8d-adc9-139048789203 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:52.232560 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545251f6-8621-4b8d-adc9-139048789203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:52.232836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545251f6-8621-4b8d-adc9-139048789203 admin admin] PATH_INFO: `/v3/projects/fe428a23adc74bbab88ff935243cc4e2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:52.234551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545251f6-8621-4b8d-adc9-139048789203 admin admin] RBAC: Authorizing `identity:get_project(project_id=fe428a23adc74bbab88ff935243cc4e2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:07:52.288178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: WARNING keystone.common.rbac_enforcer.enforcer [None req-545251f6-8621-4b8d-adc9-139048789203 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 23 22:07:52.288511 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545251f6-8621-4b8d-adc9-139048789203 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:07:52.293698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 2/5] 10.222.0.88 () {56 vars in 1207 bytes} [Tue Aug 23 22:07:52 2022] GET /identity/v3/projects/fe428a23adc74bbab88ff935243cc4e2 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:07:54.995383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78890bc9-7c69-4344-9842-93313d3f7fa3 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:54.995969 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78890bc9-7c69-4344-9842-93313d3f7fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:54.996354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78890bc9-7c69-4344-9842-93313d3f7fa3 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:55.005071 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 4/6] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:07:54 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:07:55.015945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b7d97-f6cc-4fcb-ba77-a5b153e0dfcc None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:55.016497 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b7d97-f6cc-4fcb-ba77-a5b153e0dfcc None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:55.017108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b7d97-f6cc-4fcb-ba77-a5b153e0dfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:55.052710 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-5a0b7d97-f6cc-4fcb-ba77-a5b153e0dfcc None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:07:55.055958 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5a0b7d97-f6cc-4fcb-ba77-a5b153e0dfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:07:55.070487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 3/7] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:07:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 23 22:07:55.480091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78890bc9-7c69-4344-9842-93313d3f7fa3 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:07:55.495681 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78890bc9-7c69-4344-9842-93313d3f7fa3 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:07:55.501644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ca510-db9e-4394-99b3-58baf8a46dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:07:55.502335 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ca510-db9e-4394-99b3-58baf8a46dda admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:55.502692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ca510-db9e-4394-99b3-58baf8a46dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:55.503112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ca510-db9e-4394-99b3-58baf8a46dda admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:55.513291 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ca510-db9e-4394-99b3-58baf8a46dda admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:07:55.515175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ca510-db9e-4394-99b3-58baf8a46dda admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:07:55.526361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 5/8] 10.222.0.88 () {58 vars in 1324 bytes} [Tue Aug 23 22:07:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 22:07:55.532784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b7d97-f6cc-4fcb-ba77-a5b153e0dfcc None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:07:55.543865 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b7d97-f6cc-4fcb-ba77-a5b153e0dfcc None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:07:55.548687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e3ccf-4190-4b40-b9d8-5557a8b1dce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:07:55.549432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e3ccf-4190-4b40-b9d8-5557a8b1dce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:55.549775 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e3ccf-4190-4b40-b9d8-5557a8b1dce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:55.550106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e3ccf-4190-4b40-b9d8-5557a8b1dce1 admin admin] PATH_INFO: `/v3/users/a7b11924f137447a8c88ac81ee52e9ff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:55.551560 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e3ccf-4190-4b40-b9d8-5557a8b1dce1 admin admin] RBAC: Authorizing `identity:get_user(user_id=a7b11924f137447a8c88ac81ee52e9ff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:07:55.553644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e3ccf-4190-4b40-b9d8-5557a8b1dce1 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:07:55.556043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 4/9] 10.222.0.88 () {56 vars in 1198 bytes} [Tue Aug 23 22:07:55 2022] GET /identity/v3/users/a7b11924f137447a8c88ac81ee52e9ff => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:07:57.809425 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d537-a783-4c7a-b1a5-d22c4e98cb5c None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:57.810212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d537-a783-4c7a-b1a5-d22c4e98cb5c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:57.810212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d537-a783-4c7a-b1a5-d22c4e98cb5c None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:57.811443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 6/10] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:07:57.818309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4e51c-d57e-4c45-be4f-80574daaa87e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:57.819123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4e51c-d57e-4c45-be4f-80574daaa87e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:57.819123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4e51c-d57e-4c45-be4f-80574daaa87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:57.839629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-09b4e51c-d57e-4c45-be4f-80574daaa87e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:07:57.841008 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-09b4e51c-d57e-4c45-be4f-80574daaa87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:07:57.971383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8d167-6bf0-45ba-a31b-25cbcf90644b None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:57.971687 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8d167-6bf0-45ba-a31b-25cbcf90644b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:57.971992 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8d167-6bf0-45ba-a31b-25cbcf90644b None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:57.973481 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 7/11] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:07:57.978916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 5/12] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:07:57.981469 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6653403-2e97-4a06-a58f-f6cbe2f1e89a None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:57.981837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6653403-2e97-4a06-a58f-f6cbe2f1e89a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:57.982336 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6653403-2e97-4a06-a58f-f6cbe2f1e89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:58.064881 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG passlib.handlers.bcrypt [None req-f6653403-2e97-4a06-a58f-f6cbe2f1e89a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81679) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 23 22:07:58.065538 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG passlib.handlers.bcrypt [None req-f6653403-2e97-4a06-a58f-f6cbe2f1e89a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81679) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 23 22:07:58.119503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-f6653403-2e97-4a06-a58f-f6cbe2f1e89a None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:07:58.122415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f6653403-2e97-4a06-a58f-f6cbe2f1e89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:07:58.270947 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 8/13] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:07:58.297820 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4e51c-d57e-4c45-be4f-80574daaa87e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:07:58.346527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:07:58.496900 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4e51c-d57e-4c45-be4f-80574daaa87e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:07:58.561898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c049c34-6a52-4c5b-8819-74b264d79922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:07:58.562913 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c049c34-6a52-4c5b-8819-74b264d79922 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:58.563363 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c049c34-6a52-4c5b-8819-74b264d79922 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:58.563807 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c049c34-6a52-4c5b-8819-74b264d79922 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:58.587805 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c049c34-6a52-4c5b-8819-74b264d79922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:07:58.602050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-1c049c34-6a52-4c5b-8819-74b264d79922 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:07:58.602050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:07:58.602050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:07:58.602050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:07:58.602050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:07:58.602050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:07:58.602050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:07:58.602050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:07:58.602050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:07:58.602050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:07:58.602050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:07:58.602050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:07:58.602050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:07:58.602050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:07:58.602050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:07:58.602050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:07:58.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:07:58.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:07:58.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:07:58.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:07:58.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:07:58.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:07:58.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:07:58.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:07:58.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:07:58.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:07:58.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:07:58.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:07:58.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:07:58.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:07:58.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:07:58.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:07:58.613803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:07:58.613803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:07:58.613803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:07:58.613803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:07:58.613803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:07:58.613803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:07:58.613803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:07:58.613803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:07:58.613803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:07:58.613803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:07:58.613803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:07:58.613803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:07:58.613803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 6/14] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:07:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:07:58.615059 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6653403-2e97-4a06-a58f-f6cbe2f1e89a None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:07:58.616125 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c049c34-6a52-4c5b-8819-74b264d79922 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:07:58.760076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:07:58.801925 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c049c34-6a52-4c5b-8819-74b264d79922 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:07:58.806212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6653403-2e97-4a06-a58f-f6cbe2f1e89a None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:07:58.810445 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:07:58.867935 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fa2ce-0bde-4ed2-b370-139bc00d0286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:07:58.868492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fa2ce-0bde-4ed2-b370-139bc00d0286 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:58.868796 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fa2ce-0bde-4ed2-b370-139bc00d0286 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:58.869222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fa2ce-0bde-4ed2-b370-139bc00d0286 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:58.870325 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fa2ce-0bde-4ed2-b370-139bc00d0286 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:07:58.870761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fa2ce-0bde-4ed2-b370-139bc00d0286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:07:58.872293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fa2ce-0bde-4ed2-b370-139bc00d0286 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:07:58.876472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44964213-050e-47a9-af3c-226d11bd1b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:07:58.877244 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44964213-050e-47a9-af3c-226d11bd1b1a None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:58.877577 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44964213-050e-47a9-af3c-226d11bd1b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:58.877906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44964213-050e-47a9-af3c-226d11bd1b1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:58.886285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 7/15] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:07:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:07:58.892923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fa2ce-0bde-4ed2-b370-139bc00d0286 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:07:58.973774 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44964213-050e-47a9-af3c-226d11bd1b1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:07:58.976841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44964213-050e-47a9-af3c-226d11bd1b1a None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:07:59.043076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fa2ce-0bde-4ed2-b370-139bc00d0286 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:07:59.108302 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc7f11-25b7-429d-8840-9685ebeec326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:07:59.109152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc7f11-25b7-429d-8840-9685ebeec326 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:59.110098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc7f11-25b7-429d-8840-9685ebeec326 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:59.110098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc7f11-25b7-429d-8840-9685ebeec326 None admin] PATH_INFO: `/v3/users/a7b11924f137447a8c88ac81ee52e9ff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:59.110866 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc7f11-25b7-429d-8840-9685ebeec326 None admin] RBAC: Authorizing `identity:get_user(user_id=a7b11924f137447a8c88ac81ee52e9ff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:07:59.112708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc7f11-25b7-429d-8840-9685ebeec326 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:07:59.114421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 8/16] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:07:58 2022] GET /identity/v3/users/a7b11924f137447a8c88ac81ee52e9ff => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:07:59.121841 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc7f11-25b7-429d-8840-9685ebeec326 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:07:59.141610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 9/17] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:07:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 22:07:59.145810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:07:59.147465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44964213-050e-47a9-af3c-226d11bd1b1a None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:07:59.151030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:07:59.277383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc7f11-25b7-429d-8840-9685ebeec326 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:07:59.291001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44964213-050e-47a9-af3c-226d11bd1b1a None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:07:59.329574 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0c890-583a-41a0-8fbe-657548bfef78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:07:59.330624 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0c890-583a-41a0-8fbe-657548bfef78 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:59.330824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0c890-583a-41a0-8fbe-657548bfef78 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:59.331022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0c890-583a-41a0-8fbe-657548bfef78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:59.343438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0d85d-78e3-4e31-ac3d-fd154a5c1eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:07:59.344556 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0d85d-78e3-4e31-ac3d-fd154a5c1eec None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:59.345046 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0d85d-78e3-4e31-ac3d-fd154a5c1eec None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:59.345570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0d85d-78e3-4e31-ac3d-fd154a5c1eec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:59.347629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0d85d-78e3-4e31-ac3d-fd154a5c1eec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:07:59.350351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0d85d-78e3-4e31-ac3d-fd154a5c1eec None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:07:59.353013 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0c890-583a-41a0-8fbe-657548bfef78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:07:59.354001 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 9/18] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:07:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:07:59.354567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0c890-583a-41a0-8fbe-657548bfef78 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:07:59.364677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0d85d-78e3-4e31-ac3d-fd154a5c1eec None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:07:59.376779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-82c0c890-583a-41a0-8fbe-657548bfef78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:07:59.376779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:07:59.376779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:07:59.376779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:07:59.376779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:07:59.376779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:07:59.376779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:07:59.376779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:07:59.376779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:07:59.376779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:07:59.376779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:07:59.376779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:07:59.376779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:07:59.376779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:07:59.376779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:07:59.376779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:07:59.378111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:07:59.379388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:07:59.379388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:07:59.379388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:07:59.379388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 10/19] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:07:59.390481 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0c890-583a-41a0-8fbe-657548bfef78 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:07:59.473498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:07:59.513703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:07:59.538771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0c890-583a-41a0-8fbe-657548bfef78 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:07:59.550019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0d85d-78e3-4e31-ac3d-fd154a5c1eec None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:07:59.587502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01eb954-c357-4bb2-aa9e-e6a15901f068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:07:59.588194 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01eb954-c357-4bb2-aa9e-e6a15901f068 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:59.588482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01eb954-c357-4bb2-aa9e-e6a15901f068 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:59.588763 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01eb954-c357-4bb2-aa9e-e6a15901f068 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:59.589518 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01eb954-c357-4bb2-aa9e-e6a15901f068 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:07:59.589783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01eb954-c357-4bb2-aa9e-e6a15901f068 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:07:59.591264 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01eb954-c357-4bb2-aa9e-e6a15901f068 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:07:59.597722 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c317d5-4631-43d7-80d5-d6a8fb5c9537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:07:59.598436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c317d5-4631-43d7-80d5-d6a8fb5c9537 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:07:59.598785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c317d5-4631-43d7-80d5-d6a8fb5c9537 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:07:59.599139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c317d5-4631-43d7-80d5-d6a8fb5c9537 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:07:59.599985 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c317d5-4631-43d7-80d5-d6a8fb5c9537 None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.domain.id=default, user.id=a7b11924f137447a8c88ac81ee52e9ff) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:07:59.600359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c317d5-4631-43d7-80d5-d6a8fb5c9537 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:07:59.602962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c317d5-4631-43d7-80d5-d6a8fb5c9537 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:07:59.611253 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 10/20] 10.222.0.88 () {56 vars in 1322 bytes} [Tue Aug 23 22:07:59 2022] GET /identity/v3/role_assignments?user.id=a7b11924f137447a8c88ac81ee52e9ff&scope.domain.id=default&role.id=dc5d71ab685e420697f520aa3a368177 => generated 233 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:07:59.616040 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 11/21] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:02.013767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c20e5-9c03-4594-83b2-96775d5fec8f None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:02.015464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c20e5-9c03-4594-83b2-96775d5fec8f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:02.016256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c20e5-9c03-4594-83b2-96775d5fec8f None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:02.019172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 11/22] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:02.026596 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec631c8-9e62-46ae-b7f8-f6619ce0aa7f None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:02.028114 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec631c8-9e62-46ae-b7f8-f6619ce0aa7f None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:02.028896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec631c8-9e62-46ae-b7f8-f6619ce0aa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:02.067130 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-eec631c8-9e62-46ae-b7f8-f6619ce0aa7f None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:02.068618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-eec631c8-9e62-46ae-b7f8-f6619ce0aa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:02.180543 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 12/23] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:02.838465 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c20e5-9c03-4594-83b2-96775d5fec8f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:03.000670 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c20e5-9c03-4594-83b2-96775d5fec8f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:03.051604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e334b04-a2ac-452f-b664-f42001c20a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:03.053028 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e334b04-a2ac-452f-b664-f42001c20a33 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:03.054228 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e334b04-a2ac-452f-b664-f42001c20a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:03.054715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e334b04-a2ac-452f-b664-f42001c20a33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:03.073537 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e334b04-a2ac-452f-b664-f42001c20a33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:03.075360 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e334b04-a2ac-452f-b664-f42001c20a33 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:03.080470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:03.099089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-9e334b04-a2ac-452f-b664-f42001c20a33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:08:03.099089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:03.099089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:03.099089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:03.099089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:03.099089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:03.099089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:03.099089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:03.099089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:03.099089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:03.099089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:03.099089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:03.099089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:08:03.099089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:08:03.099089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:08:03.099089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:03.102180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:08:03.103468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:08:03.103468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:08:03.103468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:03.103468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 12/24] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:03.106958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec631c8-9e62-46ae-b7f8-f6619ce0aa7f None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:03.134321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a2e37-516a-49e8-8b6c-bd9797d8f5e3 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:03.134696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a2e37-516a-49e8-8b6c-bd9797d8f5e3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:03.135255 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a2e37-516a-49e8-8b6c-bd9797d8f5e3 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:03.136779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 13/25] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:03.147069 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7f21b-57c0-43f6-9c6b-a03bb84e75cc None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:03.148893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7f21b-57c0-43f6-9c6b-a03bb84e75cc None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:03.149712 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7f21b-57c0-43f6-9c6b-a03bb84e75cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:03.193795 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:03.201170 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-94f7f21b-57c0-43f6-9c6b-a03bb84e75cc None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:03.203319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-94f7f21b-57c0-43f6-9c6b-a03bb84e75cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:03.396994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec631c8-9e62-46ae-b7f8-f6619ce0aa7f None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:03.409523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 14/26] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:08:03.500461 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ad2d3-d839-414f-bcb4-d4debcebd4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:03.501905 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ad2d3-d839-414f-bcb4-d4debcebd4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:03.502695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ad2d3-d839-414f-bcb4-d4debcebd4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:03.503516 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ad2d3-d839-414f-bcb4-d4debcebd4e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:03.504947 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ad2d3-d839-414f-bcb4-d4debcebd4e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:03.505788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ad2d3-d839-414f-bcb4-d4debcebd4e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:03.510223 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ad2d3-d839-414f-bcb4-d4debcebd4e5 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:03.553964 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 13/27] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:03.560131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7f21b-57c0-43f6-9c6b-a03bb84e75cc None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:03.687763 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797603b-d2fe-4599-b675-8f2b6328f19d None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:03.688764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797603b-d2fe-4599-b675-8f2b6328f19d None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:03.689592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797603b-d2fe-4599-b675-8f2b6328f19d None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:03.692713 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 14/28] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:08:03.704822 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40acab-5f31-4c75-b2bf-56deeadb50cf None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:03.705806 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40acab-5f31-4c75-b2bf-56deeadb50cf None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:03.706591 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40acab-5f31-4c75-b2bf-56deeadb50cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:03.766973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-9c40acab-5f31-4c75-b2bf-56deeadb50cf None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:03.769483 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-9c40acab-5f31-4c75-b2bf-56deeadb50cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:03.781768 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7f21b-57c0-43f6-9c6b-a03bb84e75cc None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:03.852438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:03.867952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82cf4b-ca4b-428c-b103-d3dea985db7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:03.869021 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82cf4b-ca4b-428c-b103-d3dea985db7a None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:03.869717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82cf4b-ca4b-428c-b103-d3dea985db7a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:03.870615 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82cf4b-ca4b-428c-b103-d3dea985db7a None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:03.872308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82cf4b-ca4b-428c-b103-d3dea985db7a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:03.878330 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82cf4b-ca4b-428c-b103-d3dea985db7a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:04.061162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 15/29] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:04.063366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c983476-018a-495a-ba35-52eae13b4dff None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:04.063883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c983476-018a-495a-ba35-52eae13b4dff None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:04.064385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c983476-018a-495a-ba35-52eae13b4dff None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:04.067189 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 16/30] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:04.087024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0df509-6552-4232-9d57-653d9eba682d None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:04.087532 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0df509-6552-4232-9d57-653d9eba682d None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:04.088030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0df509-6552-4232-9d57-653d9eba682d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:04.154935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-7a0df509-6552-4232-9d57-653d9eba682d None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:04.156912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-7a0df509-6552-4232-9d57-653d9eba682d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:04.195865 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 15/31] 10.222.0.88 () {60 vars in 1141 bytes} [Tue Aug 23 22:08:03 2022] POST /identity/v3/projects => generated 301 bytes in 636 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:08:04.205955 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82cf4b-ca4b-428c-b103-d3dea985db7a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:04.266041 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:04.396988 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 17/32] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:08:04.529794 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82cf4b-ca4b-428c-b103-d3dea985db7a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:04.609278 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b583b6-4b85-4bb7-9056-1aeb5007ffbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:04.610380 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b583b6-4b85-4bb7-9056-1aeb5007ffbc None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:04.610887 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b583b6-4b85-4bb7-9056-1aeb5007ffbc None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:04.611384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b583b6-4b85-4bb7-9056-1aeb5007ffbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:04.642722 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b583b6-4b85-4bb7-9056-1aeb5007ffbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:04.652439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-47b583b6-4b85-4bb7-9056-1aeb5007ffbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:04.652439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:04.652439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:04.652439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:04.652439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:04.652439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:04.652439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:04.652439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:04.652439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:04.652439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:04.652439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:04.652439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:04.652439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:04.652439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:04.652439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:04.652439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:04.654128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:04.654128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:04.654128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:04.654128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:04.654128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:04.654128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:04.654128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:04.654128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:04.654128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:04.654128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:04.654128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:04.654128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:04.654128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:04.654128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:04.654128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:04.654128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:04.655621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:04.655621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:04.655621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:04.655621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:04.655621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:04.655621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:04.655621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:04.655621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:04.655621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:04.655621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:04.655621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:04.655621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:04.655621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 16/33] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:08:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 458 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:04.670262 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0df509-6552-4232-9d57-653d9eba682d None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:04.820028 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0df509-6552-4232-9d57-653d9eba682d None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:04.895490 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b583b6-4b85-4bb7-9056-1aeb5007ffbc None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:04.897940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f570c-bfe6-4815-a85a-a093a86e0914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:04.898711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f570c-bfe6-4815-a85a-a093a86e0914 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:04.898982 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f570c-bfe6-4815-a85a-a093a86e0914 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:04.899256 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f570c-bfe6-4815-a85a-a093a86e0914 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:04.900164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f570c-bfe6-4815-a85a-a093a86e0914 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:04.900607 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f570c-bfe6-4815-a85a-a093a86e0914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:04.902248 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f570c-bfe6-4815-a85a-a093a86e0914 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:04.930668 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 18/34] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:04.942137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f570c-bfe6-4815-a85a-a093a86e0914 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:05.013598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:05.032657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:05.137924 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f570c-bfe6-4815-a85a-a093a86e0914 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:05.163628 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b583b6-4b85-4bb7-9056-1aeb5007ffbc None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:05.188481 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885ff99-6479-41bc-92e2-91d817bb674c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:05.189280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885ff99-6479-41bc-92e2-91d817bb674c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:05.189908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885ff99-6479-41bc-92e2-91d817bb674c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:05.190200 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885ff99-6479-41bc-92e2-91d817bb674c None admin] PATH_INFO: `/v3/users/a7b11924f137447a8c88ac81ee52e9ff` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:05.192210 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885ff99-6479-41bc-92e2-91d817bb674c None admin] RBAC: Authorizing `identity:get_user(user_id=a7b11924f137447a8c88ac81ee52e9ff)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:05.194623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885ff99-6479-41bc-92e2-91d817bb674c None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:05.197732 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 19/35] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:04 2022] GET /identity/v3/users/a7b11924f137447a8c88ac81ee52e9ff => generated 247 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:05.198756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885ff99-6479-41bc-92e2-91d817bb674c None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:05.233463 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839e1e0-5d40-4409-adf8-821dbb8770cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:05.237094 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839e1e0-5d40-4409-adf8-821dbb8770cc None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:05.237094 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839e1e0-5d40-4409-adf8-821dbb8770cc None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:05.237094 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839e1e0-5d40-4409-adf8-821dbb8770cc None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:05.237094 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839e1e0-5d40-4409-adf8-821dbb8770cc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:05.239877 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839e1e0-5d40-4409-adf8-821dbb8770cc None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:05.271944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 17/36] 10.222.0.88 () {60 vars in 1131 bytes} [Tue Aug 23 22:08:04 2022] POST /identity/v3/roles => generated 225 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:08:05.278406 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839e1e0-5d40-4409-adf8-821dbb8770cc None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:05.365194 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:05.378699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885ff99-6479-41bc-92e2-91d817bb674c None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:05.422136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929e235-aa77-4d23-9751-81c8ffd18bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:05.422774 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929e235-aa77-4d23-9751-81c8ffd18bac None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:05.423058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929e235-aa77-4d23-9751-81c8ffd18bac None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:05.423338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929e235-aa77-4d23-9751-81c8ffd18bac None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:05.424162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929e235-aa77-4d23-9751-81c8ffd18bac None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:05.425768 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929e235-aa77-4d23-9751-81c8ffd18bac None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:05.440176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:05.445975 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839e1e0-5d40-4409-adf8-821dbb8770cc None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:05.515356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64144ec7-ac59-4f34-8c39-db7ccb14af26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:05.516463 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64144ec7-ac59-4f34-8c39-db7ccb14af26 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:05.516971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64144ec7-ac59-4f34-8c39-db7ccb14af26 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:05.517594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64144ec7-ac59-4f34-8c39-db7ccb14af26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:05.519826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64144ec7-ac59-4f34-8c39-db7ccb14af26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:05.523220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64144ec7-ac59-4f34-8c39-db7ccb14af26 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:05.526558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 18/37] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:05.529138 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89629e06-8cc3-4e3b-a85f-71a96d0304f0 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:05.529628 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89629e06-8cc3-4e3b-a85f-71a96d0304f0 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:05.530126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89629e06-8cc3-4e3b-a85f-71a96d0304f0 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:05.533339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 19/38] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:05.535639 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abe48f-c054-4f4b-b704-30783f091699 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:05.536270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abe48f-c054-4f4b-b704-30783f091699 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:05.536729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abe48f-c054-4f4b-b704-30783f091699 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:05.539836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 20/39] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:05.541121 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abe48f-c054-4f4b-b704-30783f091699 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:05.568148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 20/40] 10.222.0.88 () {60 vars in 1131 bytes} [Tue Aug 23 22:08:05 2022] POST /identity/v3/roles => generated 219 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:08:05.571218 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a7d80-832c-46d8-bf4a-89017c920c68 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:05.571710 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a7d80-832c-46d8-bf4a-89017c920c68 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:05.572420 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a7d80-832c-46d8-bf4a-89017c920c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:05.599287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-b85a7d80-832c-46d8-bf4a-89017c920c68 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:05.603010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-b85a7d80-832c-46d8-bf4a-89017c920c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:05.714477 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 21/41] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:05.716771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec8bf8-b74f-4d9f-b00c-8dbc0f0e4ed7 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:05.717313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec8bf8-b74f-4d9f-b00c-8dbc0f0e4ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:05.717812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec8bf8-b74f-4d9f-b00c-8dbc0f0e4ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:05.752678 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-baec8bf8-b74f-4d9f-b00c-8dbc0f0e4ed7 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:05.755290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-baec8bf8-b74f-4d9f-b00c-8dbc0f0e4ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:05.837818 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abe48f-c054-4f4b-b704-30783f091699 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:05.898384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c066c-fe59-4e39-9ed0-a75fe2888c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:05.899680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c066c-fe59-4e39-9ed0-a75fe2888c11 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:05.900104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c066c-fe59-4e39-9ed0-a75fe2888c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:05.900530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c066c-fe59-4e39-9ed0-a75fe2888c11 None admin] PATH_INFO: `/v3/domains/default/users/a7b11924f137447a8c88ac81ee52e9ff/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:05.903255 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c066c-fe59-4e39-9ed0-a75fe2888c11 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a7b11924f137447a8c88ac81ee52e9ff, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:05.905580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c066c-fe59-4e39-9ed0-a75fe2888c11 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:05.919624 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 22/42] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:05.941351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 21/43] 10.222.0.88 () {58 vars in 1361 bytes} [Tue Aug 23 22:08:05 2022] PUT /identity/v3/domains/default/users/a7b11924f137447a8c88ac81ee52e9ff/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:08:06.210412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec8bf8-b74f-4d9f-b00c-8dbc0f0e4ed7 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:06.246125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:06.416428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec8bf8-b74f-4d9f-b00c-8dbc0f0e4ed7 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:06.461287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c066c-fe59-4e39-9ed0-a75fe2888c11 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:06.487383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bbab27-76e3-4eef-872b-ecb3f36b6408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:06.488535 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbab27-76e3-4eef-872b-ecb3f36b6408 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:06.489044 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbab27-76e3-4eef-872b-ecb3f36b6408 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:06.489619 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbab27-76e3-4eef-872b-ecb3f36b6408 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:06.491879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bbab27-76e3-4eef-872b-ecb3f36b6408 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:06.494934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bbab27-76e3-4eef-872b-ecb3f36b6408 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:06.498295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 23/44] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:06.508092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bbab27-76e3-4eef-872b-ecb3f36b6408 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:06.519405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:06.680706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:06.683173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c066c-fe59-4e39-9ed0-a75fe2888c11 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:06.712760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bbab27-76e3-4eef-872b-ecb3f36b6408 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:06.746793 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770324e-2c6a-4a62-9431-7e259f7624f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:06.747792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770324e-2c6a-4a62-9431-7e259f7624f4 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:06.748242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770324e-2c6a-4a62-9431-7e259f7624f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:06.748671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770324e-2c6a-4a62-9431-7e259f7624f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:06.749582 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770324e-2c6a-4a62-9431-7e259f7624f4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:06.751304 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770324e-2c6a-4a62-9431-7e259f7624f4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:06.773390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 22/45] 10.222.0.88 () {60 vars in 1131 bytes} [Tue Aug 23 22:08:06 2022] POST /identity/v3/roles => generated 223 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:08:06.774915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bb208-1fbe-462e-a5b2-47f474204b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:06.775576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bb208-1fbe-462e-a5b2-47f474204b32 None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:06.775863 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bb208-1fbe-462e-a5b2-47f474204b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:06.776144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bb208-1fbe-462e-a5b2-47f474204b32 None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:06.776886 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3bb208-1fbe-462e-a5b2-47f474204b32 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:06.778687 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3bb208-1fbe-462e-a5b2-47f474204b32 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:06.825384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 24/46] 10.222.0.88 () {60 vars in 1141 bytes} [Tue Aug 23 22:08:06 2022] POST /identity/v3/projects => generated 312 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:08:08.217480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba7de3-c7ea-43fd-b085-4ae1d1c8e336 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:08.218341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba7de3-c7ea-43fd-b085-4ae1d1c8e336 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:08.218517 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba7de3-c7ea-43fd-b085-4ae1d1c8e336 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:08.221034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 23/47] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:08:08.229782 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc8869-7422-49ae-812b-d219c5637c7b None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:08.230546 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc8869-7422-49ae-812b-d219c5637c7b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:08.230730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc8869-7422-49ae-812b-d219c5637c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:08.251072 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-bcfc8869-7422-49ae-812b-d219c5637c7b None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:08.252370 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-bcfc8869-7422-49ae-812b-d219c5637c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:08.354169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 25/48] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:08.845599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba7de3-c7ea-43fd-b085-4ae1d1c8e336 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:08.956979 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:08.979882 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba7de3-c7ea-43fd-b085-4ae1d1c8e336 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:09.017592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e086c3-9ec6-4780-8636-856ac88c0ad1 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:09.017898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e086c3-9ec6-4780-8636-856ac88c0ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:09.018182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e086c3-9ec6-4780-8636-856ac88c0ad1 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:09.019650 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 26/49] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:08:09.026313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558946d1-419e-422b-b897-15caaacfcb9d None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:09.026591 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558946d1-419e-422b-b897-15caaacfcb9d None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:09.026882 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558946d1-419e-422b-b897-15caaacfcb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:09.040421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f69e57-1501-4fb2-999f-51e9bdb5314e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:09.041497 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f69e57-1501-4fb2-999f-51e9bdb5314e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:09.041956 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f69e57-1501-4fb2-999f-51e9bdb5314e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:09.042410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f69e57-1501-4fb2-999f-51e9bdb5314e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:09.047073 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-558946d1-419e-422b-b897-15caaacfcb9d None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:09.048341 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-558946d1-419e-422b-b897-15caaacfcb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:09.060566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f69e57-1501-4fb2-999f-51e9bdb5314e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:09.065972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-39f69e57-1501-4fb2-999f-51e9bdb5314e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:09.065972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:09.065972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:09.065972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:09.065972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:09.065972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:09.065972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:09.065972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:09.065972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:09.065972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:09.065972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:09.065972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:09.065972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:09.065972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:09.065972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:09.065972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:09.068126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:09.068126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:09.068126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:09.068126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:09.068126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:09.068126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:09.068126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:09.068126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:09.068126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:09.068126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:09.068126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:09.068126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:09.068126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:09.068126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:09.068126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:09.068126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:09.069651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:09.069651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:09.069651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:09.069651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:09.069651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:09.069651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:09.069651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:09.069651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:09.069651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:09.069651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:09.069651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:09.069651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:09.069651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 24/50] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:08:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:09.078442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f69e57-1501-4fb2-999f-51e9bdb5314e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:09.132057 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:09.145576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 27/51] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:08:09.219339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f69e57-1501-4fb2-999f-51e9bdb5314e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:09.275906 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82107256-1af2-4e77-87e7-5973d17d5967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:09.276895 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82107256-1af2-4e77-87e7-5973d17d5967 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:09.277403 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82107256-1af2-4e77-87e7-5973d17d5967 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:09.277859 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82107256-1af2-4e77-87e7-5973d17d5967 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:09.279037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82107256-1af2-4e77-87e7-5973d17d5967 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:09.279480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82107256-1af2-4e77-87e7-5973d17d5967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:09.282109 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82107256-1af2-4e77-87e7-5973d17d5967 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:09.304896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 25/52] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:09.307466 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558946d1-419e-422b-b897-15caaacfcb9d None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:09.441133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82107256-1af2-4e77-87e7-5973d17d5967 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:09.454968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:09.485351 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558946d1-419e-422b-b897-15caaacfcb9d None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:09.536119 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361aee8-d45d-4331-8ad1-19f36380559f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:09.536951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361aee8-d45d-4331-8ad1-19f36380559f None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:09.537437 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361aee8-d45d-4331-8ad1-19f36380559f None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:09.537841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361aee8-d45d-4331-8ad1-19f36380559f None admin] PATH_INFO: `/v3/users/a7b11924f137447a8c88ac81ee52e9ff` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:09.539234 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3361aee8-d45d-4331-8ad1-19f36380559f None admin] RBAC: Authorizing `identity:get_user(user_id=a7b11924f137447a8c88ac81ee52e9ff)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:09.540857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3361aee8-d45d-4331-8ad1-19f36380559f None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:09.542584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 28/53] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:09 2022] GET /identity/v3/users/a7b11924f137447a8c88ac81ee52e9ff => generated 247 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:09.548614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361aee8-d45d-4331-8ad1-19f36380559f None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:09.587079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82107256-1af2-4e77-87e7-5973d17d5967 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:09.618675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:09.650801 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580495-f249-4ca5-899c-78823b865b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:09.651624 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580495-f249-4ca5-899c-78823b865b7e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:09.651998 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580495-f249-4ca5-899c-78823b865b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:09.652365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580495-f249-4ca5-899c-78823b865b7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:09.654096 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51580495-f249-4ca5-899c-78823b865b7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:09.656670 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51580495-f249-4ca5-899c-78823b865b7e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:09.659527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 26/54] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:09.668368 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580495-f249-4ca5-899c-78823b865b7e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:09.686896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361aee8-d45d-4331-8ad1-19f36380559f None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:09.755019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13058a0e-cc72-4df5-9b44-53ba30e85a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:09.756171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13058a0e-cc72-4df5-9b44-53ba30e85a3b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:09.757251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13058a0e-cc72-4df5-9b44-53ba30e85a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:09.757751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13058a0e-cc72-4df5-9b44-53ba30e85a3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:09.759936 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13058a0e-cc72-4df5-9b44-53ba30e85a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:09.762622 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13058a0e-cc72-4df5-9b44-53ba30e85a3b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:09.765732 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 29/55] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:09.772470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13058a0e-cc72-4df5-9b44-53ba30e85a3b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:09.801166 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:09.837199 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580495-f249-4ca5-899c-78823b865b7e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:09.898748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433cea5-456f-48ff-97f5-3831854d341a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:09.899715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433cea5-456f-48ff-97f5-3831854d341a None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:09.900154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433cea5-456f-48ff-97f5-3831854d341a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:09.900588 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433cea5-456f-48ff-97f5-3831854d341a None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:09.901724 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433cea5-456f-48ff-97f5-3831854d341a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:09.904249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433cea5-456f-48ff-97f5-3831854d341a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:09.951259 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 27/56] 10.222.0.88 () {60 vars in 1140 bytes} [Tue Aug 23 22:08:09 2022] POST /identity/v3/projects => generated 298 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:08:10.090604 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13058a0e-cc72-4df5-9b44-53ba30e85a3b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:10.144922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:10.152589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5ca26-cbf8-4e99-94fc-7e610dbd3d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:10.153406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5ca26-cbf8-4e99-94fc-7e610dbd3d8b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:10.153836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5ca26-cbf8-4e99-94fc-7e610dbd3d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:10.154299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5ca26-cbf8-4e99-94fc-7e610dbd3d8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:10.155164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5ca26-cbf8-4e99-94fc-7e610dbd3d8b None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.domain.id=default, user.id=a7b11924f137447a8c88ac81ee52e9ff) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:10.155687 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5ca26-cbf8-4e99-94fc-7e610dbd3d8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:10.157810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5ca26-cbf8-4e99-94fc-7e610dbd3d8b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:10.170339 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 30/57] 10.222.0.88 () {56 vars in 1322 bytes} [Tue Aug 23 22:08:09 2022] GET /identity/v3/role_assignments?user.id=a7b11924f137447a8c88ac81ee52e9ff&scope.domain.id=default&role.id=dc5d71ab685e420697f520aa3a368177 => generated 530 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:12.032489 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa048c7-bfd6-46b4-98af-898f921be1dd None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:12.034575 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa048c7-bfd6-46b4-98af-898f921be1dd None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:12.034575 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa048c7-bfd6-46b4-98af-898f921be1dd None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:12.035816 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 28/58] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:12.044944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c479fd-4452-4914-8adc-6d52d5098b89 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:12.045525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c479fd-4452-4914-8adc-6d52d5098b89 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:12.045960 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c479fd-4452-4914-8adc-6d52d5098b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:12.070548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-d5c479fd-4452-4914-8adc-6d52d5098b89 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:12.071863 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d5c479fd-4452-4914-8adc-6d52d5098b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:12.181596 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 31/59] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:12.738334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa048c7-bfd6-46b4-98af-898f921be1dd None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:12.908718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa048c7-bfd6-46b4-98af-898f921be1dd None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:12.958398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:12.994421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9354874-9945-4439-a21e-6039d0d17298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:12.995459 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9354874-9945-4439-a21e-6039d0d17298 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:12.995975 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9354874-9945-4439-a21e-6039d0d17298 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:12.996486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9354874-9945-4439-a21e-6039d0d17298 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:12.998590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9354874-9945-4439-a21e-6039d0d17298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:13.001424 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9354874-9945-4439-a21e-6039d0d17298 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:13.004538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 29/60] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:13.011736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c479fd-4452-4914-8adc-6d52d5098b89 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:13.200753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c479fd-4452-4914-8adc-6d52d5098b89 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:13.249931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:13.265220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfd6e3-5507-466a-98f9-2d285665f702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:13.266279 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfd6e3-5507-466a-98f9-2d285665f702 None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:13.266794 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfd6e3-5507-466a-98f9-2d285665f702 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:13.267304 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfd6e3-5507-466a-98f9-2d285665f702 None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:13.268517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bfd6e3-5507-466a-98f9-2d285665f702 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:13.271422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bfd6e3-5507-466a-98f9-2d285665f702 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:13.316015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 32/61] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:08:13 2022] POST /identity/v3/users => generated 275 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:08:17.746906 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996371b9-65e0-4470-b7ba-5c31520c67db None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:17.747458 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996371b9-65e0-4470-b7ba-5c31520c67db None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:17.747852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996371b9-65e0-4470-b7ba-5c31520c67db None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:17.750232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 30/62] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:17.777326 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac09e66-1931-4582-b9dd-efd2926d9c8c None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:17.777837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac09e66-1931-4582-b9dd-efd2926d9c8c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:17.778316 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac09e66-1931-4582-b9dd-efd2926d9c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:17.825872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-cac09e66-1931-4582-b9dd-efd2926d9c8c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:17.828486 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-cac09e66-1931-4582-b9dd-efd2926d9c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:18.031208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 33/63] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:18.118263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1e0c0-1566-423f-89dc-7e223a31c6b4 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:18.119276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1e0c0-1566-423f-89dc-7e223a31c6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:18.120073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1e0c0-1566-423f-89dc-7e223a31c6b4 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:18.123060 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 31/64] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:18.133222 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28aa1d-4f48-43c9-8569-a337bfac5c50 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:18.134221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28aa1d-4f48-43c9-8569-a337bfac5c50 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:18.135029 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28aa1d-4f48-43c9-8569-a337bfac5c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:18.180836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-3f28aa1d-4f48-43c9-8569-a337bfac5c50 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:18.183597 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3f28aa1d-4f48-43c9-8569-a337bfac5c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:18.192212 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f016de-1da2-45bf-9c13-e1819218b949 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:18.193241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f016de-1da2-45bf-9c13-e1819218b949 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:18.194998 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f016de-1da2-45bf-9c13-e1819218b949 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:18.197987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 32/65] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:18.207195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b37acf-1456-4d92-9d2d-c5697aaf6030 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:18.208106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b37acf-1456-4d92-9d2d-c5697aaf6030 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:18.208961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b37acf-1456-4d92-9d2d-c5697aaf6030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:18.267067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-06b37acf-1456-4d92-9d2d-c5697aaf6030 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:18.270322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-06b37acf-1456-4d92-9d2d-c5697aaf6030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:18.336364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 34/66] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:18.371359 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7c1f9-5bf3-484a-a506-1b0fd521875e None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:18.371894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7c1f9-5bf3-484a-a506-1b0fd521875e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:18.372325 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7c1f9-5bf3-484a-a506-1b0fd521875e None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:18.373912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 35/67] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:08:18.383171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f956fa-6cca-41d0-818d-988e92369e1d None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:18.383691 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f956fa-6cca-41d0-818d-988e92369e1d None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:18.384144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f956fa-6cca-41d0-818d-988e92369e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:18.439342 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-47f956fa-6cca-41d0-818d-988e92369e1d None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:18.441487 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-47f956fa-6cca-41d0-818d-988e92369e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:18.450265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 33/68] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:08:18.535610 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b37acf-1456-4d92-9d2d-c5697aaf6030 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:18.588942 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:18.692865 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 36/69] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:18.696534 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6d137-06c8-495c-92a1-2ee5b30233f1 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:18.696829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6d137-06c8-495c-92a1-2ee5b30233f1 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:18.697131 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6d137-06c8-495c-92a1-2ee5b30233f1 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:18.698527 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 37/70] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:18.703720 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6d137-06c8-495c-92a1-2ee5b30233f1 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:18.762276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b37acf-1456-4d92-9d2d-c5697aaf6030 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:18.832820 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:18.854638 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a20c1-e2d2-4fa6-87a2-480d639947eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:18.856953 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a20c1-e2d2-4fa6-87a2-480d639947eb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:18.857845 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a20c1-e2d2-4fa6-87a2-480d639947eb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:18.858382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a20c1-e2d2-4fa6-87a2-480d639947eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:18.863039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6d137-06c8-495c-92a1-2ee5b30233f1 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:18.880845 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a20c1-e2d2-4fa6-87a2-480d639947eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:18.893896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-655a20c1-e2d2-4fa6-87a2-480d639947eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:18.893896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:18.893896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:18.893896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:18.893896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:18.893896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:18.893896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:18.893896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:18.893896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:18.893896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:18.893896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:18.893896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:18.893896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:18.893896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:18.893896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:18.893896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:18.894909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:18.894909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:18.894909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:18.894909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:18.894909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:18.894909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:18.894909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:18.894909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:18.894909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:18.894909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:18.894909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:18.894909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:18.894909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:18.894909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:18.894909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:18.894909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:18.895703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:18.895703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:18.895703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:18.895703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:18.895703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:18.895703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:18.895703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:18.895703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:18.895703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:18.895703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:18.895703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:18.895703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:18.896832 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 34/71] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:18.902003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffd78e-441a-4131-8b16-6d3fa1d47d99 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:18.902003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffd78e-441a-4131-8b16-6d3fa1d47d99 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:18.903202 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffd78e-441a-4131-8b16-6d3fa1d47d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:18.929162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f5c4d-48f2-4d07-a5c2-b15d808d99f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:18.930627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f5c4d-48f2-4d07-a5c2-b15d808d99f6 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:18.932207 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f5c4d-48f2-4d07-a5c2-b15d808d99f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:18.932936 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f5c4d-48f2-4d07-a5c2-b15d808d99f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:18.940284 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-17ffd78e-441a-4131-8b16-6d3fa1d47d99 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:18.943638 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-17ffd78e-441a-4131-8b16-6d3fa1d47d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:18.959756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f5c4d-48f2-4d07-a5c2-b15d808d99f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:18.975354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-8c4f5c4d-48f2-4d07-a5c2-b15d808d99f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:08:18.975354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:18.975354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:18.975354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:18.975354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:18.975354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:18.975354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:18.975354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:18.975354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:18.975354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:18.975354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:18.975354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:18.975354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:18.975354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:18.975354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:18.975354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:18.976966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:18.976966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:18.976966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:18.976966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:18.976966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:18.976966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:18.976966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:18.976966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:18.976966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:18.976966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:18.976966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:18.976966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:18.976966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:18.976966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:18.976966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:18.976966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:18.978379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:18.978379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:18.978379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:18.978379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:18.978379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:18.978379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:18.978379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:18.978379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:18.978379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:18.978379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:18.978379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:08:18.978379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:08:18.978379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 38/72] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:08:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:18.981269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f5c4d-48f2-4d07-a5c2-b15d808d99f6 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:19.040592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:19.078756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 35/73] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:19.079636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffd78e-441a-4131-8b16-6d3fa1d47d99 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:19.166159 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f5c4d-48f2-4d07-a5c2-b15d808d99f6 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:19.223214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffd78e-441a-4131-8b16-6d3fa1d47d99 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:19.232287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cffe57-f401-4456-9908-89507790cd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:19.233470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cffe57-f401-4456-9908-89507790cd55 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:19.233950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cffe57-f401-4456-9908-89507790cd55 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:19.234478 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cffe57-f401-4456-9908-89507790cd55 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:19.235798 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cffe57-f401-4456-9908-89507790cd55 None admin] RBAC: Adding query filter params (name=member) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:19.236615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cffe57-f401-4456-9908-89507790cd55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:19.239545 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cffe57-f401-4456-9908-89507790cd55 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:19.255382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:19.259006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 39/74] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:08:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:19.260457 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cffe57-f401-4456-9908-89507790cd55 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:19.284698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af212e4b-d6c6-4f5a-b5b2-bc77e6c9a556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:19.285539 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af212e4b-d6c6-4f5a-b5b2-bc77e6c9a556 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:19.285825 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af212e4b-d6c6-4f5a-b5b2-bc77e6c9a556 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:19.286214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af212e4b-d6c6-4f5a-b5b2-bc77e6c9a556 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:19.303367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af212e4b-d6c6-4f5a-b5b2-bc77e6c9a556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:19.313407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-af212e4b-d6c6-4f5a-b5b2-bc77e6c9a556 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:19.313407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:19.313407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:19.313407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:19.313407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:19.313407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:19.313407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:19.313407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:19.313407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:19.313407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:19.313407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:19.313407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:19.313407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:19.313407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:19.313407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:19.313407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:19.314578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:19.314578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:19.314578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:19.314578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:19.314578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:19.314578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:19.314578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:19.314578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:19.314578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:19.314578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:19.314578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:19.314578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:19.314578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:19.314578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:19.314578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:19.314578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:19.315431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:19.315431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:19.315431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:19.315431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:19.315431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:19.315431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:19.315431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:19.315431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:19.315431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:19.315431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:19.315431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:19.315431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:19.316018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 36/75] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:08:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:19.317350 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af212e4b-d6c6-4f5a-b5b2-bc77e6c9a556 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:19.425100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:19.436773 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cffe57-f401-4456-9908-89507790cd55 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:19.471923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af212e4b-d6c6-4f5a-b5b2-bc77e6c9a556 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:19.484266 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c40d62-8632-4dc9-bade-25fde0380532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:19.485355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c40d62-8632-4dc9-bade-25fde0380532 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:19.485848 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c40d62-8632-4dc9-bade-25fde0380532 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:19.486329 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c40d62-8632-4dc9-bade-25fde0380532 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:19.487584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c40d62-8632-4dc9-bade-25fde0380532 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:19.488142 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c40d62-8632-4dc9-bade-25fde0380532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:19.490890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c40d62-8632-4dc9-bade-25fde0380532 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:19.507100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 40/76] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:08:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:19.510142 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c40d62-8632-4dc9-bade-25fde0380532 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:19.517839 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430d473-2d08-45bb-b67a-8fc37084b778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:19.518866 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430d473-2d08-45bb-b67a-8fc37084b778 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:19.519156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430d473-2d08-45bb-b67a-8fc37084b778 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:19.519492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430d473-2d08-45bb-b67a-8fc37084b778 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:19.535256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430d473-2d08-45bb-b67a-8fc37084b778 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:19.540743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-8430d473-2d08-45bb-b67a-8fc37084b778 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:19.540743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:19.540743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:19.540743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:19.540743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:19.540743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:19.540743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:19.540743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:19.540743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:19.540743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:19.540743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:19.540743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:19.540743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:19.540743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:19.540743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:19.540743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:19.542081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:19.542081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:19.542081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:19.542081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:19.542081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:19.542081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:19.542081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:19.542081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:19.542081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:19.542081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:19.542081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:19.542081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:19.542081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:19.542081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:19.542081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:19.542081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:19.542927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:19.542927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:19.542927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:19.542927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:19.542927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:19.542927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:19.542927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:19.542927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:19.542927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:19.542927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:19.542927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:19.542927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:19.542927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 37/77] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:19.543553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430d473-2d08-45bb-b67a-8fc37084b778 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:19.596589 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:19.645698 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c40d62-8632-4dc9-bade-25fde0380532 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:19.683288 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430d473-2d08-45bb-b67a-8fc37084b778 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:19.712396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8d538-1610-41b8-992f-8424f2442b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:19.713536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8d538-1610-41b8-992f-8424f2442b09 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:19.714054 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8d538-1610-41b8-992f-8424f2442b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:19.714559 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8d538-1610-41b8-992f-8424f2442b09 None admin] PATH_INFO: `/v3/users/2a83b38dedc44dd89eb854ab6be2e94d` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:19.726915 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbbf63-7b2e-42da-9af7-b7dcc18a44c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:19.727554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbbf63-7b2e-42da-9af7-b7dcc18a44c1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:19.727817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbbf63-7b2e-42da-9af7-b7dcc18a44c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:19.728094 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbbf63-7b2e-42da-9af7-b7dcc18a44c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:19.728779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbbf63-7b2e-42da-9af7-b7dcc18a44c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:19.729107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbbf63-7b2e-42da-9af7-b7dcc18a44c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:19.730558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbbf63-7b2e-42da-9af7-b7dcc18a44c1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:19.739675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 38/78] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:19.741128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbbf63-7b2e-42da-9af7-b7dcc18a44c1 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:19.746177 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8d538-1610-41b8-992f-8424f2442b09 None admin] RBAC: Authorizing `identity:get_user(user_id=2a83b38dedc44dd89eb854ab6be2e94d)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:19.747652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8d538-1610-41b8-992f-8424f2442b09 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:19.749411 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 41/79] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:19 2022] GET /identity/v3/users/2a83b38dedc44dd89eb854ab6be2e94d => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:19.750202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8d538-1610-41b8-992f-8424f2442b09 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:19.775381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:19.881846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:19.893239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbbf63-7b2e-42da-9af7-b7dcc18a44c1 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:19.904034 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8d538-1610-41b8-992f-8424f2442b09 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:19.946267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa66af-0f6d-4c43-a084-73a085a8447e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:19.946964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa66af-0f6d-4c43-a084-73a085a8447e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:19.947316 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa66af-0f6d-4c43-a084-73a085a8447e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:19.947592 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa66af-0f6d-4c43-a084-73a085a8447e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:19.949097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa66af-0f6d-4c43-a084-73a085a8447e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:19.951149 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa66af-0f6d-4c43-a084-73a085a8447e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:19.952888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 39/80] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:19.953772 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa66af-0f6d-4c43-a084-73a085a8447e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:19.955736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196eaf42-1600-4f1e-80b9-427e1f6d24e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:19.956410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eaf42-1600-4f1e-80b9-427e1f6d24e2 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:19.956921 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eaf42-1600-4f1e-80b9-427e1f6d24e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:19.957115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eaf42-1600-4f1e-80b9-427e1f6d24e2 None admin] PATH_INFO: `/v3/users/2a83b38dedc44dd89eb854ab6be2e94d` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:19.958343 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196eaf42-1600-4f1e-80b9-427e1f6d24e2 None admin] RBAC: Authorizing `identity:get_user(user_id=2a83b38dedc44dd89eb854ab6be2e94d)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:19.960536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196eaf42-1600-4f1e-80b9-427e1f6d24e2 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:19.962385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 42/81] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:19 2022] GET /identity/v3/users/2a83b38dedc44dd89eb854ab6be2e94d => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:19.963258 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196eaf42-1600-4f1e-80b9-427e1f6d24e2 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:20.100061 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:20.103214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa66af-0f6d-4c43-a084-73a085a8447e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:20.126636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196eaf42-1600-4f1e-80b9-427e1f6d24e2 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:20.152084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e5750-ba55-40e4-b0cc-48a07494ee27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:20.152911 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e5750-ba55-40e4-b0cc-48a07494ee27 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:20.153361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e5750-ba55-40e4-b0cc-48a07494ee27 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:20.153731 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e5750-ba55-40e4-b0cc-48a07494ee27 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:20.154519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e5750-ba55-40e4-b0cc-48a07494ee27 None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:20.154998 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e5750-ba55-40e4-b0cc-48a07494ee27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:20.156641 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e5750-ba55-40e4-b0cc-48a07494ee27 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:20.164934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330257f-2154-4eea-a441-dad6d46ba6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:20.165620 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330257f-2154-4eea-a441-dad6d46ba6eb None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:20.165901 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330257f-2154-4eea-a441-dad6d46ba6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:20.166177 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330257f-2154-4eea-a441-dad6d46ba6eb None admin] PATH_INFO: `/v3/users/a7b11924f137447a8c88ac81ee52e9ff` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:20.167382 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330257f-2154-4eea-a441-dad6d46ba6eb None admin] RBAC: Authorizing `identity:get_user(user_id=a7b11924f137447a8c88ac81ee52e9ff)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:20.168838 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330257f-2154-4eea-a441-dad6d46ba6eb None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:20.169361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 40/82] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:08:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:20.178905 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e5750-ba55-40e4-b0cc-48a07494ee27 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:20.179089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 43/83] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:19 2022] GET /identity/v3/users/a7b11924f137447a8c88ac81ee52e9ff => generated 247 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:20.179089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330257f-2154-4eea-a441-dad6d46ba6eb None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:20.223818 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:20.307496 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e5750-ba55-40e4-b0cc-48a07494ee27 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:20.312998 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330257f-2154-4eea-a441-dad6d46ba6eb None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:20.354209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ee260-e7de-4621-ba5c-6032b810be32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:20.354835 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee260-e7de-4621-ba5c-6032b810be32 None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:20.355120 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee260-e7de-4621-ba5c-6032b810be32 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:20.355393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee260-e7de-4621-ba5c-6032b810be32 None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:20.355785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e96a3-03fe-4655-92d1-66fcb5995f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:20.356053 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ee260-e7de-4621-ba5c-6032b810be32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:20.356438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e96a3-03fe-4655-92d1-66fcb5995f34 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:20.356726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e96a3-03fe-4655-92d1-66fcb5995f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:20.357005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e96a3-03fe-4655-92d1-66fcb5995f34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:20.357551 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ee260-e7de-4621-ba5c-6032b810be32 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:20.380227 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 44/84] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:08:20 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:08:20.381345 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ee260-e7de-4621-ba5c-6032b810be32 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:20.397996 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:20.472575 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e96a3-03fe-4655-92d1-66fcb5995f34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:20.474883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e96a3-03fe-4655-92d1-66fcb5995f34 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:20.528911 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:20.542670 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ee260-e7de-4621-ba5c-6032b810be32 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:20.611448 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bb7a0-e2f7-497f-9d9c-fafc5ff69ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:20.612521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bb7a0-e2f7-497f-9d9c-fafc5ff69ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:20.612985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bb7a0-e2f7-497f-9d9c-fafc5ff69ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:20.613566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bb7a0-e2f7-497f-9d9c-fafc5ff69ed9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:20.642385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 41/85] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:08:20.644014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e96a3-03fe-4655-92d1-66fcb5995f34 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:20.732707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bb7a0-e2f7-497f-9d9c-fafc5ff69ed9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:20.735999 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bb7a0-e2f7-497f-9d9c-fafc5ff69ed9 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:20.746841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:20.823665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e96a3-03fe-4655-92d1-66fcb5995f34 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:20.871924 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:20.885441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8935a77-6733-49d1-a59e-5b938a8175c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:20.886726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8935a77-6733-49d1-a59e-5b938a8175c6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:20.887361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8935a77-6733-49d1-a59e-5b938a8175c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:20.888029 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8935a77-6733-49d1-a59e-5b938a8175c6 None admin] PATH_INFO: `/v3/users/2a83b38dedc44dd89eb854ab6be2e94d` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:20.890361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8935a77-6733-49d1-a59e-5b938a8175c6 None admin] RBAC: Authorizing `identity:get_user(user_id=2a83b38dedc44dd89eb854ab6be2e94d)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:20.893010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8935a77-6733-49d1-a59e-5b938a8175c6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:20.897043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 42/86] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:20 2022] GET /identity/v3/users/2a83b38dedc44dd89eb854ab6be2e94d => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:20.898673 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8935a77-6733-49d1-a59e-5b938a8175c6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:20.943582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 45/87] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:08:20.946199 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bb7a0-e2f7-497f-9d9c-fafc5ff69ed9 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:21.059446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8935a77-6733-49d1-a59e-5b938a8175c6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:21.105113 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bef80-21dd-465c-8346-92b704f64d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:21.106000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bef80-21dd-465c-8346-92b704f64d39 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:21.106287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bef80-21dd-465c-8346-92b704f64d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:21.106566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bef80-21dd-465c-8346-92b704f64d39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:21.124951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bb7a0-e2f7-497f-9d9c-fafc5ff69ed9 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:21.134747 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:21.167216 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c44955-5ed1-4eca-865e-65ec3009925e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:21.167984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c44955-5ed1-4eca-865e-65ec3009925e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:21.168331 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c44955-5ed1-4eca-865e-65ec3009925e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:21.168688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c44955-5ed1-4eca-865e-65ec3009925e None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:21.170232 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c44955-5ed1-4eca-865e-65ec3009925e None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:21.171791 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c44955-5ed1-4eca-865e-65ec3009925e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:21.174132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 46/88] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:20 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:21.175155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c44955-5ed1-4eca-865e-65ec3009925e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:21.214407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:21.224870 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bef80-21dd-465c-8346-92b704f64d39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:21.226430 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bef80-21dd-465c-8346-92b704f64d39 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:21.311973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c44955-5ed1-4eca-865e-65ec3009925e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:21.357627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f226e4-4adf-45f2-8555-8b021196290e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:21.358503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f226e4-4adf-45f2-8555-8b021196290e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:21.358785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f226e4-4adf-45f2-8555-8b021196290e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:21.359065 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f226e4-4adf-45f2-8555-8b021196290e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:21.376128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 43/89] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:08:21.377845 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bef80-21dd-465c-8346-92b704f64d39 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:21.415448 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:21.452768 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f226e4-4adf-45f2-8555-8b021196290e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:21.455028 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f226e4-4adf-45f2-8555-8b021196290e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:21.518804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:21.529358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bef80-21dd-465c-8346-92b704f64d39 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:21.583136 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86254fc6-116e-4aec-907d-12d91236e614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:21.585030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86254fc6-116e-4aec-907d-12d91236e614 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:21.585492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86254fc6-116e-4aec-907d-12d91236e614 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:21.586448 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86254fc6-116e-4aec-907d-12d91236e614 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:21.588235 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86254fc6-116e-4aec-907d-12d91236e614 None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:21.590252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86254fc6-116e-4aec-907d-12d91236e614 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:21.593195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 44/90] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:21 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:21.594255 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86254fc6-116e-4aec-907d-12d91236e614 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:21.608475 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 47/91] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:08:21.610070 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f226e4-4adf-45f2-8555-8b021196290e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:21.717464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:21.764051 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86254fc6-116e-4aec-907d-12d91236e614 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:21.769499 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f226e4-4adf-45f2-8555-8b021196290e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:21.813199 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5dbe27-4d1a-4e3c-bc69-cc0f30fc84b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:21.814157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5dbe27-4d1a-4e3c-bc69-cc0f30fc84b3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:21.814331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5dbe27-4d1a-4e3c-bc69-cc0f30fc84b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:21.814515 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5dbe27-4d1a-4e3c-bc69-cc0f30fc84b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:21.815275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5dbe27-4d1a-4e3c-bc69-cc0f30fc84b3 None admin] RBAC: Adding query filter params (role.id=fad7ba55962b40459b40dcc2bad395b2, scope.project.id=709dbd6287a240f3b6044bf6d6e63334, user.id=2a83b38dedc44dd89eb854ab6be2e94d) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:21.815587 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5dbe27-4d1a-4e3c-bc69-cc0f30fc84b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:21.817401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5dbe27-4d1a-4e3c-bc69-cc0f30fc84b3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:21.829616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 45/92] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:21 2022] GET /identity/v3/role_assignments?user.id=2a83b38dedc44dd89eb854ab6be2e94d&scope.project.id=709dbd6287a240f3b6044bf6d6e63334&role.id=fad7ba55962b40459b40dcc2bad395b2 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:21.829616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5dbe27-4d1a-4e3c-bc69-cc0f30fc84b3 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:21.841536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8117ae9-fe0b-421c-9649-59065d82c702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:21.842390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8117ae9-fe0b-421c-9649-59065d82c702 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:21.842809 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8117ae9-fe0b-421c-9649-59065d82c702 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:21.843092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8117ae9-fe0b-421c-9649-59065d82c702 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:21.850333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8117ae9-fe0b-421c-9649-59065d82c702 None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:21.852380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8117ae9-fe0b-421c-9649-59065d82c702 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:21.856577 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 48/93] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:21 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:21.858654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8117ae9-fe0b-421c-9649-59065d82c702 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:21.913922 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:22.046450 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5dbe27-4d1a-4e3c-bc69-cc0f30fc84b3 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:22.074201 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8117ae9-fe0b-421c-9649-59065d82c702 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:22.098389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdb7a2-2b5d-42dc-a064-1803df28be85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:22.098962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdb7a2-2b5d-42dc-a064-1803df28be85 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:22.099869 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdb7a2-2b5d-42dc-a064-1803df28be85 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:22.100325 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdb7a2-2b5d-42dc-a064-1803df28be85 None admin] PATH_INFO: `/v3/projects/4e75717e9b354d478ba30ddbc5c5d1c1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:22.102799 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bdb7a2-2b5d-42dc-a064-1803df28be85 None admin] RBAC: Authorizing `identity:get_project(project_id=4e75717e9b354d478ba30ddbc5c5d1c1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:22.104345 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bdb7a2-2b5d-42dc-a064-1803df28be85 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:22.106332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 46/94] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:21 2022] GET /identity/v3/projects/4e75717e9b354d478ba30ddbc5c5d1c1 => generated 312 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:22.110943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdb7a2-2b5d-42dc-a064-1803df28be85 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:22.165864 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18b0e0-9f00-4631-869f-d3ee7f583450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:22.166510 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18b0e0-9f00-4631-869f-d3ee7f583450 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:22.166800 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18b0e0-9f00-4631-869f-d3ee7f583450 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:22.167086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18b0e0-9f00-4631-869f-d3ee7f583450 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:22.167854 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18b0e0-9f00-4631-869f-d3ee7f583450 None admin] RBAC: Adding query filter params (role.id=3c32f171f34345e4958d31e3940092d6, scope.project.id=709dbd6287a240f3b6044bf6d6e63334, user.id=2a83b38dedc44dd89eb854ab6be2e94d) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:22.168166 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18b0e0-9f00-4631-869f-d3ee7f583450 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:22.169737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18b0e0-9f00-4631-869f-d3ee7f583450 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:22.192099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 49/95] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:21 2022] GET /identity/v3/role_assignments?user.id=2a83b38dedc44dd89eb854ab6be2e94d&scope.project.id=709dbd6287a240f3b6044bf6d6e63334&role.id=3c32f171f34345e4958d31e3940092d6 => generated 259 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:22.193085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18b0e0-9f00-4631-869f-d3ee7f583450 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:22.218093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:22.326673 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:22.349826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdb7a2-2b5d-42dc-a064-1803df28be85 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:22.468729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87990ff-b4ed-4b89-8462-ee215b41ad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:22.470375 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87990ff-b4ed-4b89-8462-ee215b41ad0b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:22.470871 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87990ff-b4ed-4b89-8462-ee215b41ad0b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:22.472625 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87990ff-b4ed-4b89-8462-ee215b41ad0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:22.473454 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87990ff-b4ed-4b89-8462-ee215b41ad0b None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=709dbd6287a240f3b6044bf6d6e63334, user.id=a7b11924f137447a8c88ac81ee52e9ff) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:22.473770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87990ff-b4ed-4b89-8462-ee215b41ad0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:22.476196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87990ff-b4ed-4b89-8462-ee215b41ad0b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:22.487640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18b0e0-9f00-4631-869f-d3ee7f583450 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:22.491542 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 47/96] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:22 2022] GET /identity/v3/role_assignments?user.id=a7b11924f137447a8c88ac81ee52e9ff&scope.project.id=709dbd6287a240f3b6044bf6d6e63334&role.id=dc5d71ab685e420697f520aa3a368177 => generated 259 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:22.560846 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953db37-c208-417d-9b23-3664f6c3f5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:22.561978 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953db37-c208-417d-9b23-3664f6c3f5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:22.562473 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953db37-c208-417d-9b23-3664f6c3f5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:22.562978 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953db37-c208-417d-9b23-3664f6c3f5e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:22.564416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953db37-c208-417d-9b23-3664f6c3f5e6 None admin] RBAC: Adding query filter params (role.id=fad7ba55962b40459b40dcc2bad395b2, scope.project.id=4e75717e9b354d478ba30ddbc5c5d1c1, user.id=2a83b38dedc44dd89eb854ab6be2e94d) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:22.564974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953db37-c208-417d-9b23-3664f6c3f5e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:22.568684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953db37-c208-417d-9b23-3664f6c3f5e6 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:22.585914 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 50/97] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:22 2022] GET /identity/v3/role_assignments?user.id=2a83b38dedc44dd89eb854ab6be2e94d&scope.project.id=4e75717e9b354d478ba30ddbc5c5d1c1&role.id=fad7ba55962b40459b40dcc2bad395b2 => generated 259 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:23.125672 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad9b5e-389f-4e6f-bda9-a9a4fc832552 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:23.126356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad9b5e-389f-4e6f-bda9-a9a4fc832552 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:23.126356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad9b5e-389f-4e6f-bda9-a9a4fc832552 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:23.127904 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 48/98] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:23.134465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f83e55-ae14-4c50-a804-df27e409a3ca None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:23.135427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f83e55-ae14-4c50-a804-df27e409a3ca None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:23.135913 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f83e55-ae14-4c50-a804-df27e409a3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:23.162654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-95f83e55-ae14-4c50-a804-df27e409a3ca None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:23.164248 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-95f83e55-ae14-4c50-a804-df27e409a3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:23.259728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:23.265973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 51/99] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:23.352530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ed99b-d6f7-47a3-a5d5-7d2e7bab770e None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:23.353549 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ed99b-d6f7-47a3-a5d5-7d2e7bab770e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:23.354346 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ed99b-d6f7-47a3-a5d5-7d2e7bab770e None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:23.357437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 49/100] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:23.368779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc220ef-db68-4118-9594-475393afb3ad None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:23.369752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc220ef-db68-4118-9594-475393afb3ad None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:23.370537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc220ef-db68-4118-9594-475393afb3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:23.405205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-6cc220ef-db68-4118-9594-475393afb3ad None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:23.407362 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6cc220ef-db68-4118-9594-475393afb3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:23.548958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 52/101] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:23.851012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ed99b-d6f7-47a3-a5d5-7d2e7bab770e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:24.008223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ed99b-d6f7-47a3-a5d5-7d2e7bab770e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:24.078830 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a909b7-7fad-4fb7-b131-5feb38e11cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:24.079820 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a909b7-7fad-4fb7-b131-5feb38e11cbf None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:24.080333 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a909b7-7fad-4fb7-b131-5feb38e11cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:24.080833 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a909b7-7fad-4fb7-b131-5feb38e11cbf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:24.110460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a909b7-7fad-4fb7-b131-5feb38e11cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:24.121117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-78a909b7-7fad-4fb7-b131-5feb38e11cbf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:08:24.121117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:24.121117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:24.121117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:24.121117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:24.121117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:24.121117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:24.121117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:24.121117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:24.121117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:24.121117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:24.121117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:24.121117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:24.121117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:24.121117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:24.121117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:24.122758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:24.122758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:24.122758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:24.122758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:24.122758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:24.122758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:24.122758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:24.122758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:24.122758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:24.122758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:24.122758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:24.122758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:24.122758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:24.122758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:24.122758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:24.122758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:24.124929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:24.124929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:24.124929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:24.124929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:24.124929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:24.124929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:24.124929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:24.124929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:24.124929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:24.124929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:24.124929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:08:24.124929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:24.124929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 50/102] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:24.135193 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc220ef-db68-4118-9594-475393afb3ad None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:24.343350 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc220ef-db68-4118-9594-475393afb3ad None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:24.382783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a909b7-7fad-4fb7-b131-5feb38e11cbf None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:24.400367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:24.421554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3856b-7e17-47a4-b92a-17f808efe798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:24.422704 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3856b-7e17-47a4-b92a-17f808efe798 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:24.423210 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3856b-7e17-47a4-b92a-17f808efe798 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:24.423707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3856b-7e17-47a4-b92a-17f808efe798 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:24.424991 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3856b-7e17-47a4-b92a-17f808efe798 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:24.426715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3856b-7e17-47a4-b92a-17f808efe798 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:24.435713 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3856b-7e17-47a4-b92a-17f808efe798 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:24.464687 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 53/103] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:08:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:24.476455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3856b-7e17-47a4-b92a-17f808efe798 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:24.480412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:24.852688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a909b7-7fad-4fb7-b131-5feb38e11cbf None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:24.918044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698aa6f-13d4-4a38-951c-a8d07976a784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:24.919152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698aa6f-13d4-4a38-951c-a8d07976a784 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:24.919663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698aa6f-13d4-4a38-951c-a8d07976a784 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:24.920167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698aa6f-13d4-4a38-951c-a8d07976a784 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:24.922241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698aa6f-13d4-4a38-951c-a8d07976a784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:24.925272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698aa6f-13d4-4a38-951c-a8d07976a784 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:24.929373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 51/104] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 547 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:24.933108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3856b-7e17-47a4-b92a-17f808efe798 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:24.938671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698aa6f-13d4-4a38-951c-a8d07976a784 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:25.033072 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc3fcb-20d5-4b1a-b102-b5283235ae43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:25.034181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc3fcb-20d5-4b1a-b102-b5283235ae43 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:25.034688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc3fcb-20d5-4b1a-b102-b5283235ae43 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:25.037599 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc3fcb-20d5-4b1a-b102-b5283235ae43 None admin] PATH_INFO: `/v3/users/ac799727c48b4492940172c9924e103f` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:25.091675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc3fcb-20d5-4b1a-b102-b5283235ae43 None admin] RBAC: Authorizing `identity:get_user(user_id=ac799727c48b4492940172c9924e103f)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:25.095593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc3fcb-20d5-4b1a-b102-b5283235ae43 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:25.105525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 54/105] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:24 2022] GET /identity/v3/users/ac799727c48b4492940172c9924e103f => generated 289 bytes in 625 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:25.111121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc3fcb-20d5-4b1a-b102-b5283235ae43 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:25.277458 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:25.289897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:25.330100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698aa6f-13d4-4a38-951c-a8d07976a784 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:25.407586 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f34bf-db0d-4abb-ba1d-24622ac9e7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:25.408648 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f34bf-db0d-4abb-ba1d-24622ac9e7e6 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:25.417240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f34bf-db0d-4abb-ba1d-24622ac9e7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:25.417938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f34bf-db0d-4abb-ba1d-24622ac9e7e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:25.419196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f34bf-db0d-4abb-ba1d-24622ac9e7e6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:25.422049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f34bf-db0d-4abb-ba1d-24622ac9e7e6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:25.451368 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc3fcb-20d5-4b1a-b102-b5283235ae43 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:25.499410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 52/106] 10.222.0.88 () {60 vars in 1141 bytes} [Tue Aug 23 22:08:24 2022] POST /identity/v3/projects => generated 302 bytes in 562 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:08:25.542360 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca33363-c540-4202-8fd3-1b6cf652c55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:25.543427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca33363-c540-4202-8fd3-1b6cf652c55f None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:25.543940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca33363-c540-4202-8fd3-1b6cf652c55f None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:25.544423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca33363-c540-4202-8fd3-1b6cf652c55f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:25.731900 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca33363-c540-4202-8fd3-1b6cf652c55f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:25.734815 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca33363-c540-4202-8fd3-1b6cf652c55f None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:25.875170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:25.999573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 55/107] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 889 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:08:26.009752 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f34bf-db0d-4abb-ba1d-24622ac9e7e6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:26.249909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f34bf-db0d-4abb-ba1d-24622ac9e7e6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:26.319589 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac0faf-ffab-4a08-9bb7-5667c6138a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:26.320637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac0faf-ffab-4a08-9bb7-5667c6138a13 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:26.321119 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac0faf-ffab-4a08-9bb7-5667c6138a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:26.321610 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac0faf-ffab-4a08-9bb7-5667c6138a13 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:26.323837 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac0faf-ffab-4a08-9bb7-5667c6138a13 None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:26.326452 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac0faf-ffab-4a08-9bb7-5667c6138a13 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:26.330073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 53/108] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:26 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:26.338588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca33363-c540-4202-8fd3-1b6cf652c55f None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:26.423260 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445f30b-a290-4c54-ab80-f0be56e8146b None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:26.424043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445f30b-a290-4c54-ab80-f0be56e8146b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:26.424732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445f30b-a290-4c54-ab80-f0be56e8146b None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:26.427608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 54/109] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:26.434847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f24149-2e28-4400-b3f5-0d5761883803 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:26.435742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f24149-2e28-4400-b3f5-0d5761883803 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:26.436495 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f24149-2e28-4400-b3f5-0d5761883803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:26.489099 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d4f24149-2e28-4400-b3f5-0d5761883803 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:26.492610 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d4f24149-2e28-4400-b3f5-0d5761883803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:26.497620 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:26.639698 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca33363-c540-4202-8fd3-1b6cf652c55f None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:26.680176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 55/110] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:26.736385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e99de-6bdc-4852-9226-491a14994f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:26.737392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e99de-6bdc-4852-9226-491a14994f22 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:26.737831 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e99de-6bdc-4852-9226-491a14994f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:26.738264 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e99de-6bdc-4852-9226-491a14994f22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:26.739238 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e99de-6bdc-4852-9226-491a14994f22 None admin] RBAC: Adding query filter params (role.id=8d5b36826f914e958bd80fc21156e286, scope.project.id=709dbd6287a240f3b6044bf6d6e63334, user.id=ac799727c48b4492940172c9924e103f) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:26.739930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e99de-6bdc-4852-9226-491a14994f22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:26.741777 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e99de-6bdc-4852-9226-491a14994f22 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:26.754219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 56/111] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:26 2022] GET /identity/v3/role_assignments?user.id=ac799727c48b4492940172c9924e103f&scope.project.id=709dbd6287a240f3b6044bf6d6e63334&role.id=8d5b36826f914e958bd80fc21156e286 => generated 259 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:26.836245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56f669-5d9f-4b4b-879c-9409f0960895 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:26.837133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56f669-5d9f-4b4b-879c-9409f0960895 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:26.837886 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56f669-5d9f-4b4b-879c-9409f0960895 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:26.841218 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 56/112] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:26.847805 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb8a4e-cb43-47ad-bc9f-fc47e51d199f None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:26.848623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb8a4e-cb43-47ad-bc9f-fc47e51d199f None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:26.849929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb8a4e-cb43-47ad-bc9f-fc47e51d199f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:26.893985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-dafb8a4e-cb43-47ad-bc9f-fc47e51d199f None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:26.896380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-dafb8a4e-cb43-47ad-bc9f-fc47e51d199f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:26.909971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c7e81-7906-4dc2-a1c0-9e7784ef1536 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:26.910774 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c7e81-7906-4dc2-a1c0-9e7784ef1536 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:26.912013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c7e81-7906-4dc2-a1c0-9e7784ef1536 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:26.915023 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 57/113] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:08:26.921961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e4dec-f28d-4b8f-82bf-1152d0cd3a3b None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:26.922739 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e4dec-f28d-4b8f-82bf-1152d0cd3a3b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:26.923423 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e4dec-f28d-4b8f-82bf-1152d0cd3a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:26.961511 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:26.989046 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-991e4dec-f28d-4b8f-82bf-1152d0cd3a3b None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:26.991980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-991e4dec-f28d-4b8f-82bf-1152d0cd3a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:27.088803 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 57/114] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:27.179992 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 58/115] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:08:27.268137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb8a4e-cb43-47ad-bc9f-fc47e51d199f None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:27.504183 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb8a4e-cb43-47ad-bc9f-fc47e51d199f None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:27.579318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625a0df-aa5c-4811-95a9-f1e0a508d8da None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:27.580646 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625a0df-aa5c-4811-95a9-f1e0a508d8da None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:27.581365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625a0df-aa5c-4811-95a9-f1e0a508d8da None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:27.584446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfad34-9a83-4a36-875f-1f2ebdebcb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:27.585098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 59/116] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:27.589181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfad34-9a83-4a36-875f-1f2ebdebcb8b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:27.589181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfad34-9a83-4a36-875f-1f2ebdebcb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:27.589181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfad34-9a83-4a36-875f-1f2ebdebcb8b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:27.592065 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd57e69-9ac3-429d-b81c-4338f554b9c8 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:27.592793 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd57e69-9ac3-429d-b81c-4338f554b9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:27.593507 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd57e69-9ac3-429d-b81c-4338f554b9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:27.610973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfad34-9a83-4a36-875f-1f2ebdebcb8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:27.626027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-efbfad34-9a83-4a36-875f-1f2ebdebcb8b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:08:27.626027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:27.626027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:27.626027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:27.626027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:27.626027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:27.626027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:27.626027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:27.626027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:27.626027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:27.626027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:27.626027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:27.626027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:27.626027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:27.626027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:27.626027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:27.628006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:27.628006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:27.628006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:27.628006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:27.628006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:27.628006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:27.628006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:27.628006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:27.628006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:27.628006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:27.628006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:27.628006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:27.628006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:27.628006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:27.628006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:27.628006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:27.629751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:27.629751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:27.629751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:27.629751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:27.629751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:27.629751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:27.629751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:27.629751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:27.629751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:27.629751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:27.629751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:08:27.629751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:08:27.629751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 58/117] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:08:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:27.643148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfad34-9a83-4a36-875f-1f2ebdebcb8b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:27.656247 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:27.658193 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ecd57e69-9ac3-429d-b81c-4338f554b9c8 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:27.660438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ecd57e69-9ac3-429d-b81c-4338f554b9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:27.845186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:27.856888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 60/118] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:27.859560 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd57e69-9ac3-429d-b81c-4338f554b9c8 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:27.904260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfad34-9a83-4a36-875f-1f2ebdebcb8b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:28.031643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349143f-47f4-4125-914e-b0a2ec623909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:28.036531 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349143f-47f4-4125-914e-b0a2ec623909 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:28.038162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349143f-47f4-4125-914e-b0a2ec623909 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:28.040245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349143f-47f4-4125-914e-b0a2ec623909 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:28.044485 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349143f-47f4-4125-914e-b0a2ec623909 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:28.045472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349143f-47f4-4125-914e-b0a2ec623909 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:28.048200 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349143f-47f4-4125-914e-b0a2ec623909 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:28.084483 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 59/119] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:08:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:28.086434 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349143f-47f4-4125-914e-b0a2ec623909 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:28.119849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd57e69-9ac3-429d-b81c-4338f554b9c8 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:28.218179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b2358-7679-4ffe-bb21-e470067b5f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:28.218847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b2358-7679-4ffe-bb21-e470067b5f68 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:28.219133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b2358-7679-4ffe-bb21-e470067b5f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:28.219410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b2358-7679-4ffe-bb21-e470067b5f68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:28.233108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b2358-7679-4ffe-bb21-e470067b5f68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:28.236603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:28.240373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-c97b2358-7679-4ffe-bb21-e470067b5f68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:28.240373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:28.240373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:28.240373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:28.240373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:28.240373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:28.240373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:28.240373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:28.240373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:28.240373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:28.240373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:28.240373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:28.240373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:28.240373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:28.240373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:28.240373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:28.242214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:28.242214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:28.242214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:28.242214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:28.242214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:28.242214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:28.242214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:28.242214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:28.242214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:28.242214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:28.242214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:28.242214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:28.242214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:28.242214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:28.242214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:28.242214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:28.243634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:28.243634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:28.243634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:28.243634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:28.243634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:28.243634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:28.243634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:28.243634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:28.243634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:28.243634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:28.243634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:28.243634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:28.243634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 61/120] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:08:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:28.248586 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b2358-7679-4ffe-bb21-e470067b5f68 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:28.326820 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349143f-47f4-4125-914e-b0a2ec623909 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:28.345767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:28.392435 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b2358-7679-4ffe-bb21-e470067b5f68 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:28.412245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e126a-1a5f-461d-bdb6-52868f530771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:28.414778 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e126a-1a5f-461d-bdb6-52868f530771 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:28.415053 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e126a-1a5f-461d-bdb6-52868f530771 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:28.415333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e126a-1a5f-461d-bdb6-52868f530771 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:28.440376 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e126a-1a5f-461d-bdb6-52868f530771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:28.441751 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb2cad-a190-4c3f-aa27-b60a618d6661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:28.442502 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb2cad-a190-4c3f-aa27-b60a618d6661 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:28.442899 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb2cad-a190-4c3f-aa27-b60a618d6661 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:28.443399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb2cad-a190-4c3f-aa27-b60a618d6661 None admin] PATH_INFO: `/v3/users/2a83b38dedc44dd89eb854ab6be2e94d` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:28.444830 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb2cad-a190-4c3f-aa27-b60a618d6661 None admin] RBAC: Authorizing `identity:get_user(user_id=2a83b38dedc44dd89eb854ab6be2e94d)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:28.446538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb2cad-a190-4c3f-aa27-b60a618d6661 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:28.448815 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 62/121] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:28 2022] GET /identity/v3/users/2a83b38dedc44dd89eb854ab6be2e94d => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:28.452007 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb2cad-a190-4c3f-aa27-b60a618d6661 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:28.457174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-320e126a-1a5f-461d-bdb6-52868f530771 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:28.457174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:28.457174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:28.457174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:28.457174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:28.457174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:28.457174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:28.457174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:28.457174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:28.457174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:28.457174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:28.457174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:28.457174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:28.457174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:28.457174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:28.457174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:28.459252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:28.459252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:28.459252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:28.459252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:28.459252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:28.459252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:28.459252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:28.459252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:28.459252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:28.459252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:28.459252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:28.459252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:28.459252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:28.459252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:28.459252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:28.459252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:28.460593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:28.460593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:28.460593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:28.460593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:28.460593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:28.460593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:28.460593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:28.460593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:28.460593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:28.460593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:28.460593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:28.460593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:08:28.465494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 60/122] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:28.470797 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e126a-1a5f-461d-bdb6-52868f530771 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:28.629542 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb2cad-a190-4c3f-aa27-b60a618d6661 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:28.682489 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:28.683759 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e126a-1a5f-461d-bdb6-52868f530771 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:28.701186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968af54-072a-44b4-ad62-b760bd3f833a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:28.702636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968af54-072a-44b4-ad62-b760bd3f833a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:28.703496 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968af54-072a-44b4-ad62-b760bd3f833a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:28.704278 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968af54-072a-44b4-ad62-b760bd3f833a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:28.725473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968af54-072a-44b4-ad62-b760bd3f833a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:28.736540 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:28.739220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-2968af54-072a-44b4-ad62-b760bd3f833a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:28.739220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:28.739220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:28.739220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:28.739220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:28.739220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:28.739220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:28.739220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:28.739220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:28.739220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:28.739220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:28.739220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:28.739220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:28.739220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:28.739220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:28.739220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:28.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:28.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:28.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:28.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:28.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:28.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:28.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:28.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:28.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:28.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:28.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:28.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:28.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:28.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:28.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:28.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:28.742952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:28.742952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:28.742952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:28.742952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:28.742952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:28.742952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:28.742952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:28.742952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:28.742952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:28.742952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:28.742952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:28.742952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:28.748296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 63/123] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:28.750238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968af54-072a-44b4-ad62-b760bd3f833a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:28.761508 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387d61a-55dd-494f-b6ad-92195706ec31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:28.763867 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387d61a-55dd-494f-b6ad-92195706ec31 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:28.764621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387d61a-55dd-494f-b6ad-92195706ec31 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:28.765350 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387d61a-55dd-494f-b6ad-92195706ec31 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:28.766784 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387d61a-55dd-494f-b6ad-92195706ec31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:28.767573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387d61a-55dd-494f-b6ad-92195706ec31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:28.770403 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387d61a-55dd-494f-b6ad-92195706ec31 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:28.792327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 61/124] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:28.794433 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387d61a-55dd-494f-b6ad-92195706ec31 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:28.969025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968af54-072a-44b4-ad62-b760bd3f833a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:28.999383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387d61a-55dd-494f-b6ad-92195706ec31 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:29.036019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2045e2-18c1-426f-b680-81fde60b35cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:29.038226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2045e2-18c1-426f-b680-81fde60b35cb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:29.038954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2045e2-18c1-426f-b680-81fde60b35cb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:29.039688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2045e2-18c1-426f-b680-81fde60b35cb None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:29.042019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2045e2-18c1-426f-b680-81fde60b35cb None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:29.045024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2045e2-18c1-426f-b680-81fde60b35cb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:29.048926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 64/125] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:28 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:29.050869 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2045e2-18c1-426f-b680-81fde60b35cb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:29.074256 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e05a7-c000-44dd-bad1-6eac137201f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:29.075279 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e05a7-c000-44dd-bad1-6eac137201f6 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:29.075756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e05a7-c000-44dd-bad1-6eac137201f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:29.076231 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e05a7-c000-44dd-bad1-6eac137201f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:29.078134 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e05a7-c000-44dd-bad1-6eac137201f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:29.078675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e05a7-c000-44dd-bad1-6eac137201f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:29.081569 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e05a7-c000-44dd-bad1-6eac137201f6 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:29.109162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 62/126] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:08:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:29.109162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e05a7-c000-44dd-bad1-6eac137201f6 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:29.171119 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:29.201780 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:29.242001 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2045e2-18c1-426f-b680-81fde60b35cb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:29.271618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e05a7-c000-44dd-bad1-6eac137201f6 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:29.288923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d13dd-3f8f-4d0c-a51e-cdcb029288eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:29.289944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d13dd-3f8f-4d0c-a51e-cdcb029288eb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:29.290368 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d13dd-3f8f-4d0c-a51e-cdcb029288eb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:29.290702 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d13dd-3f8f-4d0c-a51e-cdcb029288eb None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:29.291466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d13dd-3f8f-4d0c-a51e-cdcb029288eb None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:29.291937 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d13dd-3f8f-4d0c-a51e-cdcb029288eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:29.293579 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d13dd-3f8f-4d0c-a51e-cdcb029288eb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:29.305501 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 65/127] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:08:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:29.306922 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d13dd-3f8f-4d0c-a51e-cdcb029288eb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:29.323872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f34ddd-e5e0-4973-8862-a8366e93d6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:29.324940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f34ddd-e5e0-4973-8862-a8366e93d6bc None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:29.325420 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f34ddd-e5e0-4973-8862-a8366e93d6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:29.325877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f34ddd-e5e0-4973-8862-a8366e93d6bc None admin] PATH_INFO: `/v3/users/a7b11924f137447a8c88ac81ee52e9ff` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:29.327163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f34ddd-e5e0-4973-8862-a8366e93d6bc None admin] RBAC: Authorizing `identity:get_user(user_id=a7b11924f137447a8c88ac81ee52e9ff)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:29.328701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f34ddd-e5e0-4973-8862-a8366e93d6bc None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:29.335695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 63/128] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:29 2022] GET /identity/v3/users/a7b11924f137447a8c88ac81ee52e9ff => generated 247 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:29.335695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f34ddd-e5e0-4973-8862-a8366e93d6bc None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:29.464582 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d13dd-3f8f-4d0c-a51e-cdcb029288eb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:29.481272 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f34ddd-e5e0-4973-8862-a8366e93d6bc None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:29.485416 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:29.506457 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:29.509771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93559a8b-6a26-40aa-adea-c3d11f3d6757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:29.510470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93559a8b-6a26-40aa-adea-c3d11f3d6757 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:29.510766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93559a8b-6a26-40aa-adea-c3d11f3d6757 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:29.511031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93559a8b-6a26-40aa-adea-c3d11f3d6757 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334/users/2a83b38dedc44dd89eb854ab6be2e94d/roles/3c32f171f34345e4958d31e3940092d6` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:29.513778 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93559a8b-6a26-40aa-adea-c3d11f3d6757 None admin] RBAC: Authorizing `identity:create_grant(project_id=709dbd6287a240f3b6044bf6d6e63334, user_id=2a83b38dedc44dd89eb854ab6be2e94d, role_id=3c32f171f34345e4958d31e3940092d6)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:29.515256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93559a8b-6a26-40aa-adea-c3d11f3d6757 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:29.530555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 66/129] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:08:29 2022] PUT /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334/users/2a83b38dedc44dd89eb854ab6be2e94d/roles/3c32f171f34345e4958d31e3940092d6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:08:29.539155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372054f-c149-4ca8-90da-2374936c7e5c None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:29.539821 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372054f-c149-4ca8-90da-2374936c7e5c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:29.539923 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56340051-c92d-486e-b4c9-c22fc6b4fead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:29.540478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372054f-c149-4ca8-90da-2374936c7e5c None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:29.540967 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56340051-c92d-486e-b4c9-c22fc6b4fead None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:29.541500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56340051-c92d-486e-b4c9-c22fc6b4fead None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:29.541994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56340051-c92d-486e-b4c9-c22fc6b4fead None admin] PATH_INFO: `/v3/users/2a83b38dedc44dd89eb854ab6be2e94d` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:29.542987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 67/130] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:29 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:29.544156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372054f-c149-4ca8-90da-2374936c7e5c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:29.550900 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56340051-c92d-486e-b4c9-c22fc6b4fead None admin] RBAC: Authorizing `identity:get_user(user_id=2a83b38dedc44dd89eb854ab6be2e94d)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:29.555042 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56340051-c92d-486e-b4c9-c22fc6b4fead None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:29.565681 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 64/131] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:29 2022] GET /identity/v3/users/2a83b38dedc44dd89eb854ab6be2e94d => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:29.568876 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56340051-c92d-486e-b4c9-c22fc6b4fead None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:29.710797 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372054f-c149-4ca8-90da-2374936c7e5c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:29.773808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b838ba-352d-4e7d-a968-3e9c265f5250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:29.774451 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b838ba-352d-4e7d-a968-3e9c265f5250 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:29.774732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b838ba-352d-4e7d-a968-3e9c265f5250 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:29.775062 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b838ba-352d-4e7d-a968-3e9c265f5250 None admin] PATH_INFO: `/v3/users/2a83b38dedc44dd89eb854ab6be2e94d` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:29.777193 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b838ba-352d-4e7d-a968-3e9c265f5250 None admin] RBAC: Authorizing `identity:get_user(user_id=2a83b38dedc44dd89eb854ab6be2e94d)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:29.778977 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b838ba-352d-4e7d-a968-3e9c265f5250 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:29.780968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 68/132] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:29 2022] GET /identity/v3/users/2a83b38dedc44dd89eb854ab6be2e94d => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:29.785427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8d8c0-76f5-4913-91cd-0acba80dcfcc None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:29.785427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8d8c0-76f5-4913-91cd-0acba80dcfcc None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:29.785427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8d8c0-76f5-4913-91cd-0acba80dcfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:29.808457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fab8d8c0-76f5-4913-91cd-0acba80dcfcc None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:29.809938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fab8d8c0-76f5-4913-91cd-0acba80dcfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:29.812598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56340051-c92d-486e-b4c9-c22fc6b4fead None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:29.863776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:29.879296 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0199b-c287-4d83-ae63-dada20848354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:29.880089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0199b-c287-4d83-ae63-dada20848354 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:29.880372 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0199b-c287-4d83-ae63-dada20848354 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:29.880766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0199b-c287-4d83-ae63-dada20848354 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:29.882028 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0199b-c287-4d83-ae63-dada20848354 None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:29.883507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0199b-c287-4d83-ae63-dada20848354 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:29.885397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 65/133] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:29 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:29.887979 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0199b-c287-4d83-ae63-dada20848354 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:29.916345 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 69/134] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:29.918254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8d8c0-76f5-4913-91cd-0acba80dcfcc None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:30.009175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:30.104286 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0199b-c287-4d83-ae63-dada20848354 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:30.141306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8d8c0-76f5-4913-91cd-0acba80dcfcc None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:30.174912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d4929-b7dc-4282-acb9-1ecb5bec0849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:30.176468 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d4929-b7dc-4282-acb9-1ecb5bec0849 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:30.177804 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d4929-b7dc-4282-acb9-1ecb5bec0849 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:30.178553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d4929-b7dc-4282-acb9-1ecb5bec0849 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:30.181224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d4929-b7dc-4282-acb9-1ecb5bec0849 None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:30.184474 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d4929-b7dc-4282-acb9-1ecb5bec0849 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:30.188246 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 66/135] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:29 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:30.190713 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91557da-67f8-40d5-bdd9-29c48bf2fef2 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:30.191712 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91557da-67f8-40d5-bdd9-29c48bf2fef2 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:30.192430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91557da-67f8-40d5-bdd9-29c48bf2fef2 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:30.195798 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 67/136] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:30.197527 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91557da-67f8-40d5-bdd9-29c48bf2fef2 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:30.228426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cc517-28a4-482a-b87d-4ee3f7db65d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:30.229703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cc517-28a4-482a-b87d-4ee3f7db65d9 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:30.230281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cc517-28a4-482a-b87d-4ee3f7db65d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:30.230794 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cc517-28a4-482a-b87d-4ee3f7db65d9 None admin] PATH_INFO: `/v3/projects/4e75717e9b354d478ba30ddbc5c5d1c1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:30.232842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cc517-28a4-482a-b87d-4ee3f7db65d9 None admin] RBAC: Authorizing `identity:get_project(project_id=4e75717e9b354d478ba30ddbc5c5d1c1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:30.235773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cc517-28a4-482a-b87d-4ee3f7db65d9 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:30.239076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 70/137] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:29 2022] GET /identity/v3/projects/4e75717e9b354d478ba30ddbc5c5d1c1 => generated 312 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:30.241740 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cc517-28a4-482a-b87d-4ee3f7db65d9 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:30.328255 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:30.356415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91557da-67f8-40d5-bdd9-29c48bf2fef2 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:30.395203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:30.417468 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864dd4e-0e4b-44fa-a897-8fb17a2aa4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:30.418676 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864dd4e-0e4b-44fa-a897-8fb17a2aa4b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:30.419175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864dd4e-0e4b-44fa-a897-8fb17a2aa4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:30.419694 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864dd4e-0e4b-44fa-a897-8fb17a2aa4b3 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334/users/a7b11924f137447a8c88ac81ee52e9ff/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:30.423243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864dd4e-0e4b-44fa-a897-8fb17a2aa4b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=709dbd6287a240f3b6044bf6d6e63334, user_id=a7b11924f137447a8c88ac81ee52e9ff, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:30.426152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864dd4e-0e4b-44fa-a897-8fb17a2aa4b3 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:30.431084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cc517-28a4-482a-b87d-4ee3f7db65d9 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:30.454665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 68/138] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:08:30 2022] PUT /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334/users/a7b11924f137447a8c88ac81ee52e9ff/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:08:30.456670 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f56a9-d2ec-425a-a94b-5b4540df7576 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:30.457206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f56a9-d2ec-425a-a94b-5b4540df7576 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:30.457737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f56a9-d2ec-425a-a94b-5b4540df7576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:30.491863 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-4b5f56a9-d2ec-425a-a94b-5b4540df7576 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:30.494188 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4b5f56a9-d2ec-425a-a94b-5b4540df7576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:30.513936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b7b43-4099-4bb2-9c48-e226cfb48be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:30.515486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b7b43-4099-4bb2-9c48-e226cfb48be6 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:30.516113 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b7b43-4099-4bb2-9c48-e226cfb48be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:30.516833 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b7b43-4099-4bb2-9c48-e226cfb48be6 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334/users/2a83b38dedc44dd89eb854ab6be2e94d/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:30.521513 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b7b43-4099-4bb2-9c48-e226cfb48be6 None admin] RBAC: Authorizing `identity:create_grant(project_id=709dbd6287a240f3b6044bf6d6e63334, user_id=2a83b38dedc44dd89eb854ab6be2e94d, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:30.525188 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b7b43-4099-4bb2-9c48-e226cfb48be6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:30.553839 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 71/139] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:08:30 2022] PUT /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334/users/2a83b38dedc44dd89eb854ab6be2e94d/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:08:30.560295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b7b43-4099-4bb2-9c48-e226cfb48be6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:30.671062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 69/140] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:30.673381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f56a9-d2ec-425a-a94b-5b4540df7576 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:30.828041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b7b43-4099-4bb2-9c48-e226cfb48be6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:30.834762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:30.906795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96612eb-24b8-4e7b-b0d5-14388706354b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:30.908233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96612eb-24b8-4e7b-b0d5-14388706354b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:30.912631 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96612eb-24b8-4e7b-b0d5-14388706354b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:30.914577 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96612eb-24b8-4e7b-b0d5-14388706354b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:30.953557 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96612eb-24b8-4e7b-b0d5-14388706354b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:30.964097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-f96612eb-24b8-4e7b-b0d5-14388706354b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:08:30.964097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:30.964097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:30.964097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:30.964097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:30.964097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:30.964097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:30.964097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:30.964097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:30.964097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:30.964097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:30.964097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:30.964097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:30.964097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:30.964097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:30.964097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:30.966104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:30.966104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:30.966104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:30.966104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:30.966104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:30.966104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:30.966104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:30.966104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:30.966104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:30.966104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:30.966104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:30.966104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:30.966104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:30.966104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:30.966104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:30.966104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:30.967669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:30.967669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:30.967669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:30.967669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:30.967669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:30.967669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:30.967669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:30.967669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:30.967669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:30.967669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:30.967669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:08:30.967669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:30.969848 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 72/141] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:30.983448 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96612eb-24b8-4e7b-b0d5-14388706354b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:31.023939 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f56a9-d2ec-425a-a94b-5b4540df7576 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:31.075893 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:31.186383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f983f-7591-4d00-b89a-c845e2f8a6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:31.187585 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f983f-7591-4d00-b89a-c845e2f8a6be None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:31.188091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f983f-7591-4d00-b89a-c845e2f8a6be None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:31.188598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f983f-7591-4d00-b89a-c845e2f8a6be None admin] PATH_INFO: `/v3/projects/4e75717e9b354d478ba30ddbc5c5d1c1/users/2a83b38dedc44dd89eb854ab6be2e94d/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:31.199567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f983f-7591-4d00-b89a-c845e2f8a6be None admin] RBAC: Authorizing `identity:create_grant(project_id=4e75717e9b354d478ba30ddbc5c5d1c1, user_id=2a83b38dedc44dd89eb854ab6be2e94d, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:31.202624 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f983f-7591-4d00-b89a-c845e2f8a6be None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:31.241989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 70/142] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:08:30 2022] PUT /identity/v3/projects/4e75717e9b354d478ba30ddbc5c5d1c1/users/2a83b38dedc44dd89eb854ab6be2e94d/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:08:31.243618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f983f-7591-4d00-b89a-c845e2f8a6be None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:31.292101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96612eb-24b8-4e7b-b0d5-14388706354b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:31.379475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee2bdf-63d0-46c6-983c-90ecde299b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:31.381170 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee2bdf-63d0-46c6-983c-90ecde299b6c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:31.383051 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee2bdf-63d0-46c6-983c-90ecde299b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:31.383994 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee2bdf-63d0-46c6-983c-90ecde299b6c None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:31.385611 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee2bdf-63d0-46c6-983c-90ecde299b6c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:31.386483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee2bdf-63d0-46c6-983c-90ecde299b6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:31.390487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee2bdf-63d0-46c6-983c-90ecde299b6c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:31.409307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 73/143] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:08:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:31.420203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee2bdf-63d0-46c6-983c-90ecde299b6c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:31.509048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:31.565236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f983f-7591-4d00-b89a-c845e2f8a6be None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:31.639428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae14710-3513-4c94-9854-153f8192b8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:31.641147 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae14710-3513-4c94-9854-153f8192b8fb None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:31.642225 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae14710-3513-4c94-9854-153f8192b8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:31.643073 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae14710-3513-4c94-9854-153f8192b8fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:31.646411 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae14710-3513-4c94-9854-153f8192b8fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:31.649030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee2bdf-63d0-46c6-983c-90ecde299b6c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:31.649754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae14710-3513-4c94-9854-153f8192b8fb None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:31.654169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 71/144] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:31.664626 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae14710-3513-4c94-9854-153f8192b8fb None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:31.698702 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ba939-6911-4631-b847-5dab7b4c94dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:31.699360 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ba939-6911-4631-b847-5dab7b4c94dd None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:31.699646 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ba939-6911-4631-b847-5dab7b4c94dd None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:31.699914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ba939-6911-4631-b847-5dab7b4c94dd None admin] PATH_INFO: `/v3/users/ac799727c48b4492940172c9924e103f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:31.706000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ba939-6911-4631-b847-5dab7b4c94dd None admin] RBAC: Authorizing `identity:get_user(user_id=ac799727c48b4492940172c9924e103f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:31.707781 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ba939-6911-4631-b847-5dab7b4c94dd None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:31.710826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 74/145] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:31 2022] GET /identity/v3/users/ac799727c48b4492940172c9924e103f => generated 289 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:31.720106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ba939-6911-4631-b847-5dab7b4c94dd None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:31.805197 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:31.951324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:31.984817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ba939-6911-4631-b847-5dab7b4c94dd None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:31.994099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae14710-3513-4c94-9854-153f8192b8fb None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:32.091144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac675e8d-9871-4dd2-80bc-0df93fa5db87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:32.092121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac675e8d-9871-4dd2-80bc-0df93fa5db87 None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:32.092121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac675e8d-9871-4dd2-80bc-0df93fa5db87 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:32.092382 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac675e8d-9871-4dd2-80bc-0df93fa5db87 None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:32.092857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac675e8d-9871-4dd2-80bc-0df93fa5db87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:32.097165 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac675e8d-9871-4dd2-80bc-0df93fa5db87 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:32.119128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee07944-c4fc-40d2-a00a-5115a7e9dc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:32.120674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee07944-c4fc-40d2-a00a-5115a7e9dc71 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:32.121442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee07944-c4fc-40d2-a00a-5115a7e9dc71 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:32.122161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee07944-c4fc-40d2-a00a-5115a7e9dc71 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:32.130556 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee07944-c4fc-40d2-a00a-5115a7e9dc71 None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:32.133829 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee07944-c4fc-40d2-a00a-5115a7e9dc71 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:32.137475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 75/146] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:31 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:32.141915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 72/147] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:08:31 2022] POST /identity/v3/users => generated 283 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:08:32.147348 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee07944-c4fc-40d2-a00a-5115a7e9dc71 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:32.390891 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee07944-c4fc-40d2-a00a-5115a7e9dc71 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:32.461768 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e4f18-a74a-4edf-97a8-61c1a11651d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:32.462470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e4f18-a74a-4edf-97a8-61c1a11651d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:32.462818 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e4f18-a74a-4edf-97a8-61c1a11651d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:32.463100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e4f18-a74a-4edf-97a8-61c1a11651d5 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334/users/ac799727c48b4492940172c9924e103f/roles/8d5b36826f914e958bd80fc21156e286` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:32.465290 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e4f18-a74a-4edf-97a8-61c1a11651d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=709dbd6287a240f3b6044bf6d6e63334, user_id=ac799727c48b4492940172c9924e103f, role_id=8d5b36826f914e958bd80fc21156e286)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:32.466808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e4f18-a74a-4edf-97a8-61c1a11651d5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:32.491927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 76/148] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:08:32 2022] PUT /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334/users/ac799727c48b4492940172c9924e103f/roles/8d5b36826f914e958bd80fc21156e286 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:08:33.004169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d0321-13bb-4f12-ac8c-29ba3f079b4b None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:33.006337 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d0321-13bb-4f12-ac8c-29ba3f079b4b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:33.006337 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d0321-13bb-4f12-ac8c-29ba3f079b4b None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:33.006337 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 73/149] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:33.014450 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effaa01b-6e27-400e-a1d5-b9c311a78497 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:33.014947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effaa01b-6e27-400e-a1d5-b9c311a78497 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:33.015382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effaa01b-6e27-400e-a1d5-b9c311a78497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:33.052258 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-effaa01b-6e27-400e-a1d5-b9c311a78497 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:33.054155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-effaa01b-6e27-400e-a1d5-b9c311a78497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:33.075917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:33.187698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 77/150] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:33.894973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d0321-13bb-4f12-ac8c-29ba3f079b4b None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:33.917555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93842fbd-61f1-46f9-ac02-ee18b8f406ba None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:33.918301 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93842fbd-61f1-46f9-ac02-ee18b8f406ba None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:33.918771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93842fbd-61f1-46f9-ac02-ee18b8f406ba None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:33.920536 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 78/151] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:08:33.926596 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead6a4c-03d5-4695-8cfc-9d6df2aa58d0 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:33.927763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead6a4c-03d5-4695-8cfc-9d6df2aa58d0 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:33.928239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead6a4c-03d5-4695-8cfc-9d6df2aa58d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:33.964434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-aead6a4c-03d5-4695-8cfc-9d6df2aa58d0 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:33.967121 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-aead6a4c-03d5-4695-8cfc-9d6df2aa58d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:34.139456 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 79/152] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:34.159849 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:34.183056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d0321-13bb-4f12-ac8c-29ba3f079b4b None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:34.255234 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e838c2e-aa4b-4c1a-a5f8-3d53ff880b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:34.256347 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e838c2e-aa4b-4c1a-a5f8-3d53ff880b69 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:34.256848 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e838c2e-aa4b-4c1a-a5f8-3d53ff880b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:34.257338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e838c2e-aa4b-4c1a-a5f8-3d53ff880b69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:34.279724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e838c2e-aa4b-4c1a-a5f8-3d53ff880b69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:34.285414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-9e838c2e-aa4b-4c1a-a5f8-3d53ff880b69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:08:34.285414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:34.285414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:34.285414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:34.285414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:34.285414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:34.285414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:34.285414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:34.285414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:34.285414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:34.285414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:34.285414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:34.285414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:34.285414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:34.285414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:34.285414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:34.286929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:34.286929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:34.286929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:34.286929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:34.286929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:34.286929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:34.286929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:34.286929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:34.286929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:34.286929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:34.286929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:34.286929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:34.286929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:34.286929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:34.286929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:34.286929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:34.288228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:34.288228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:34.288228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:34.288228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:34.288228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:34.288228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:34.288228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:34.288228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:34.288228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:34.288228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:34.288228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:08:34.288228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:08:34.288228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 74/153] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:08:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:34.293745 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead6a4c-03d5-4695-8cfc-9d6df2aa58d0 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:34.403181 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:34.442490 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead6a4c-03d5-4695-8cfc-9d6df2aa58d0 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:34.496595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6cb05f-f97e-4bf9-99b3-33360a89843f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:34.497332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cb05f-f97e-4bf9-99b3-33360a89843f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:34.497616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cb05f-f97e-4bf9-99b3-33360a89843f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:34.497889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cb05f-f97e-4bf9-99b3-33360a89843f None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:34.498589 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6cb05f-f97e-4bf9-99b3-33360a89843f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:34.498887 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6cb05f-f97e-4bf9-99b3-33360a89843f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:34.500355 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6cb05f-f97e-4bf9-99b3-33360a89843f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:34.511761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 80/154] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:08:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:34.517508 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e838c2e-aa4b-4c1a-a5f8-3d53ff880b69 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:34.599758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac1f69-4855-4a29-a49b-ac80336b31c3 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:34.600193 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac1f69-4855-4a29-a49b-ac80336b31c3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:34.600629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac1f69-4855-4a29-a49b-ac80336b31c3 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:34.603063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 81/155] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:34.620037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0f9ec-f82d-4eaf-b77d-602adfd758d1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:34.620473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0f9ec-f82d-4eaf-b77d-602adfd758d1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:34.620909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0f9ec-f82d-4eaf-b77d-602adfd758d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:34.670261 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-05f0f9ec-f82d-4eaf-b77d-602adfd758d1 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:34.671400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-05f0f9ec-f82d-4eaf-b77d-602adfd758d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:34.701621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e838c2e-aa4b-4c1a-a5f8-3d53ff880b69 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:34.805978 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f41bb-19f6-4d80-a1e8-b67b941f7626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:34.806635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f41bb-19f6-4d80-a1e8-b67b941f7626 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:34.806919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f41bb-19f6-4d80-a1e8-b67b941f7626 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:34.807193 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f41bb-19f6-4d80-a1e8-b67b941f7626 None admin] PATH_INFO: `/v3/users/2a83b38dedc44dd89eb854ab6be2e94d` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:34.808410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f41bb-19f6-4d80-a1e8-b67b941f7626 None admin] RBAC: Authorizing `identity:get_user(user_id=2a83b38dedc44dd89eb854ab6be2e94d)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:34.810058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f41bb-19f6-4d80-a1e8-b67b941f7626 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:34.811753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 75/156] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:34 2022] GET /identity/v3/users/2a83b38dedc44dd89eb854ab6be2e94d => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:34.826302 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f41bb-19f6-4d80-a1e8-b67b941f7626 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:34.827710 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 82/157] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:08:34.875889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:34.979922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f41bb-19f6-4d80-a1e8-b67b941f7626 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:35.017581 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0f9ec-f82d-4eaf-b77d-602adfd758d1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:35.032740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e5f46-c244-44c0-b088-d2e4b8386c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:35.032740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e5f46-c244-44c0-b088-d2e4b8386c1a None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:35.033425 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e5f46-c244-44c0-b088-d2e4b8386c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:35.033798 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e5f46-c244-44c0-b088-d2e4b8386c1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:35.178609 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:35.210549 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e5f46-c244-44c0-b088-d2e4b8386c1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:35.212400 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e5f46-c244-44c0-b088-d2e4b8386c1a None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:35.251270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0f9ec-f82d-4eaf-b77d-602adfd758d1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:35.277746 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:35.336495 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb60bb-5eca-4255-a9a3-42f5773d2364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:35.337563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb60bb-5eca-4255-a9a3-42f5773d2364 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:35.338063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb60bb-5eca-4255-a9a3-42f5773d2364 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:35.338533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb60bb-5eca-4255-a9a3-42f5773d2364 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:35.363427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb60bb-5eca-4255-a9a3-42f5773d2364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:35.377540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-68eb60bb-5eca-4255-a9a3-42f5773d2364 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:35.377540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:35.377540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:35.377540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:35.377540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:35.377540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:35.377540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:35.377540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:35.377540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:35.377540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:35.377540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:35.377540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:35.377540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:35.377540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:35.377540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:35.377540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:35.378474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:35.378474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:35.378474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:35.378474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:35.378474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:35.378474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:35.378474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:35.378474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:35.378474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:35.378474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:35.378474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:35.378474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:35.378474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:35.378474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:35.378474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:35.378474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:35.379289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:35.379289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:35.379289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:35.379289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:35.379289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:35.379289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:35.379289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:35.379289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:35.379289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:35.379289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:35.379289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:35.379289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:35.379289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 83/158] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:08:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:35.390159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb60bb-5eca-4255-a9a3-42f5773d2364 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:35.419391 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 76/159] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:08:35.425980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e5f46-c244-44c0-b088-d2e4b8386c1a None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:35.539379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e5f46-c244-44c0-b088-d2e4b8386c1a None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:35.589624 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:35.590208 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb60bb-5eca-4255-a9a3-42f5773d2364 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:35.618189 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2886f-8809-4543-89bf-445520777a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:35.619246 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2886f-8809-4543-89bf-445520777a62 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:35.619733 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2886f-8809-4543-89bf-445520777a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:35.620202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2886f-8809-4543-89bf-445520777a62 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:35.622813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2886f-8809-4543-89bf-445520777a62 None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:35.624629 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2886f-8809-4543-89bf-445520777a62 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:35.628678 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 77/160] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:35 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:35.649732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:35.651241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2886f-8809-4543-89bf-445520777a62 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:35.675380 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a5c58-e962-4d90-ae74-154b8e6e22ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:35.676346 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a5c58-e962-4d90-ae74-154b8e6e22ff None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:35.677484 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a5c58-e962-4d90-ae74-154b8e6e22ff None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:35.677942 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a5c58-e962-4d90-ae74-154b8e6e22ff None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:35.678859 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a5c58-e962-4d90-ae74-154b8e6e22ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:35.681348 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a5c58-e962-4d90-ae74-154b8e6e22ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:35.683194 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a5c58-e962-4d90-ae74-154b8e6e22ff None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:35.705245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 84/161] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:35.711158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a5c58-e962-4d90-ae74-154b8e6e22ff None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:35.893480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2886f-8809-4543-89bf-445520777a62 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:35.919578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a5c58-e962-4d90-ae74-154b8e6e22ff None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:35.962224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46711ebc-6243-4720-9cb3-cf4ce418ce63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:35.963220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46711ebc-6243-4720-9cb3-cf4ce418ce63 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:35.963699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46711ebc-6243-4720-9cb3-cf4ce418ce63 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:35.964165 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46711ebc-6243-4720-9cb3-cf4ce418ce63 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:35.993855 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46711ebc-6243-4720-9cb3-cf4ce418ce63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:36.004245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-46711ebc-6243-4720-9cb3-cf4ce418ce63 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:36.004245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:36.004245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:36.004245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:36.004245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:36.004245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:36.004245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:36.004245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:36.004245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:36.004245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:36.004245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:36.004245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:36.004245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:36.004245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:36.004245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:36.004245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:36.005307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:36.005307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:36.005307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:36.005307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:36.005307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:36.005307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:36.005307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:36.005307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:36.005307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:36.005307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:36.005307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:36.005307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:36.005307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:36.005307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:36.005307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:36.005307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:36.006523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:36.006523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:36.006523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:36.006523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:36.006523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:36.006523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:36.006523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:36.006523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:36.006523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:36.006523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:36.006523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:36.006523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:36.008413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbdda59-04dc-4fce-be05-f52113c40e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:36.009826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdda59-04dc-4fce-be05-f52113c40e63 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:36.010331 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdda59-04dc-4fce-be05-f52113c40e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:36.010816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdda59-04dc-4fce-be05-f52113c40e63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:36.011768 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbdda59-04dc-4fce-be05-f52113c40e63 None admin] RBAC: Adding query filter params (role.id=3c32f171f34345e4958d31e3940092d6, scope.project.id=709dbd6287a240f3b6044bf6d6e63334, user.id=2a83b38dedc44dd89eb854ab6be2e94d) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:36.012266 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbdda59-04dc-4fce-be05-f52113c40e63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:36.015441 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbdda59-04dc-4fce-be05-f52113c40e63 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:36.015968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 85/162] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:36.017046 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46711ebc-6243-4720-9cb3-cf4ce418ce63 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:36.027432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 78/163] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:35 2022] GET /identity/v3/role_assignments?user.id=2a83b38dedc44dd89eb854ab6be2e94d&scope.project.id=709dbd6287a240f3b6044bf6d6e63334&role.id=3c32f171f34345e4958d31e3940092d6 => generated 608 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:36.028203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbdda59-04dc-4fce-be05-f52113c40e63 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:36.097618 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:36.143325 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46711ebc-6243-4720-9cb3-cf4ce418ce63 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:36.145501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:36.186553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be191bf-74b4-464b-8759-ec72217cd2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:36.188270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be191bf-74b4-464b-8759-ec72217cd2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:36.188809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be191bf-74b4-464b-8759-ec72217cd2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:36.189343 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be191bf-74b4-464b-8759-ec72217cd2d3 None admin] PATH_INFO: `/v3/users/a7b11924f137447a8c88ac81ee52e9ff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:36.191290 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be191bf-74b4-464b-8759-ec72217cd2d3 None admin] RBAC: Authorizing `identity:get_user(user_id=a7b11924f137447a8c88ac81ee52e9ff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:36.194938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be191bf-74b4-464b-8759-ec72217cd2d3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:36.197175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 86/164] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:36 2022] GET /identity/v3/users/a7b11924f137447a8c88ac81ee52e9ff => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:36.198671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b5433-8f94-4539-8f7f-11f95923b8dc None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:36.199321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b5433-8f94-4539-8f7f-11f95923b8dc None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:36.199798 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b5433-8f94-4539-8f7f-11f95923b8dc None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:36.201361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 87/165] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:36.211395 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff76886-2ab1-43af-a0a0-00b64306df9e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:36.212073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff76886-2ab1-43af-a0a0-00b64306df9e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:36.212604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff76886-2ab1-43af-a0a0-00b64306df9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:36.247025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6ff76886-2ab1-43af-a0a0-00b64306df9e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:36.248741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6ff76886-2ab1-43af-a0a0-00b64306df9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:36.312022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbdda59-04dc-4fce-be05-f52113c40e63 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:36.399826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28721a-4bec-495f-97b2-f8b173778a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:36.400468 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28721a-4bec-495f-97b2-f8b173778a5e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:36.400749 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28721a-4bec-495f-97b2-f8b173778a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:36.401027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28721a-4bec-495f-97b2-f8b173778a5e None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:36.401767 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28721a-4bec-495f-97b2-f8b173778a5e None admin] RBAC: Adding query filter params (name=member) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:36.402074 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28721a-4bec-495f-97b2-f8b173778a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:36.403525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28721a-4bec-495f-97b2-f8b173778a5e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:36.429032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 79/166] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:08:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:36.430468 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28721a-4bec-495f-97b2-f8b173778a5e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:36.458770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 88/167] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:36.460081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a61655-e3c7-454e-88f5-7e5c05afa0c9 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:36.460370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a61655-e3c7-454e-88f5-7e5c05afa0c9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:36.460647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a61655-e3c7-454e-88f5-7e5c05afa0c9 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:36.462137 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 89/168] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:36.462711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a61655-e3c7-454e-88f5-7e5c05afa0c9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:36.605694 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:36.671815 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28721a-4bec-495f-97b2-f8b173778a5e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:36.727880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:36.758684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a61655-e3c7-454e-88f5-7e5c05afa0c9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:36.761708 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab56d8-56d0-4bf1-b9fa-79ca124735fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:36.762350 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab56d8-56d0-4bf1-b9fa-79ca124735fb None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:36.762638 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab56d8-56d0-4bf1-b9fa-79ca124735fb None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:36.762917 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab56d8-56d0-4bf1-b9fa-79ca124735fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:36.859361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cbb4f7-2ba4-4ee9-99d3-4c2e2321c1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:36.860027 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbb4f7-2ba4-4ee9-99d3-4c2e2321c1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:36.860311 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbb4f7-2ba4-4ee9-99d3-4c2e2321c1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:36.860583 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbb4f7-2ba4-4ee9-99d3-4c2e2321c1c4 None admin] PATH_INFO: `/v3/users/2a83b38dedc44dd89eb854ab6be2e94d` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:36.866699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cbb4f7-2ba4-4ee9-99d3-4c2e2321c1c4 None admin] RBAC: Authorizing `identity:get_user(user_id=2a83b38dedc44dd89eb854ab6be2e94d)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:36.869682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cbb4f7-2ba4-4ee9-99d3-4c2e2321c1c4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:36.873196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 90/169] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:36 2022] GET /identity/v3/users/2a83b38dedc44dd89eb854ab6be2e94d => generated 275 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:36.874772 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f2a28-fc36-4739-9efd-02611235fc6c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:36.875190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f2a28-fc36-4739-9efd-02611235fc6c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:36.875469 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f2a28-fc36-4739-9efd-02611235fc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:36.911947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-4e9f2a28-fc36-4739-9efd-02611235fc6c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:36.914657 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4e9f2a28-fc36-4739-9efd-02611235fc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:37.008460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab56d8-56d0-4bf1-b9fa-79ca124735fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:37.010098 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab56d8-56d0-4bf1-b9fa-79ca124735fb None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:37.024296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 91/170] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:37.025335 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f2a28-fc36-4739-9efd-02611235fc6c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:37.091860 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:37.149520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f2a28-fc36-4739-9efd-02611235fc6c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:37.227292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952e4ae-59de-4056-bb29-d81059dc4b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:37.229714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952e4ae-59de-4056-bb29-d81059dc4b6a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:37.230462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952e4ae-59de-4056-bb29-d81059dc4b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:37.231848 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952e4ae-59de-4056-bb29-d81059dc4b6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:37.399596 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 80/171] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 970 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:08:37.401861 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab56d8-56d0-4bf1-b9fa-79ca124735fb None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:37.426014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952e4ae-59de-4056-bb29-d81059dc4b6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:37.428107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952e4ae-59de-4056-bb29-d81059dc4b6a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:37.442472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:37.530620 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:37.592270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 92/172] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:08:37.592903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952e4ae-59de-4056-bb29-d81059dc4b6a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:37.617587 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab56d8-56d0-4bf1-b9fa-79ca124735fb None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:37.691762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21241971-9715-4bd8-9465-dc197c38d649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:37.692947 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21241971-9715-4bd8-9465-dc197c38d649 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:37.693645 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21241971-9715-4bd8-9465-dc197c38d649 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:37.694151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21241971-9715-4bd8-9465-dc197c38d649 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:37.711525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21241971-9715-4bd8-9465-dc197c38d649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:37.720416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-21241971-9715-4bd8-9465-dc197c38d649 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:37.720416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:37.720416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:37.720416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:37.720416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:37.720416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:37.720416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:37.720416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:37.720416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:37.720416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:37.720416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:37.720416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:37.720416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:37.720416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:37.720416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:37.720416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:37.725049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:37.725049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:37.725049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:37.725049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:37.725049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:37.725049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:37.725049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:37.725049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:37.725049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:37.725049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:37.725049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:37.725049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:37.725049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:37.725049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:37.725049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:37.725049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:37.726728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:37.726728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:37.726728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:37.726728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:37.726728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:37.726728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:37.726728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:37.726728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:37.726728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:37.726728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:37.726728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:37.726728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:08:37.732152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 81/173] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:37.734050 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21241971-9715-4bd8-9465-dc197c38d649 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:37.761388 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952e4ae-59de-4056-bb29-d81059dc4b6a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:37.855649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf2c21-07de-413d-988a-b83a113042e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:37.856750 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf2c21-07de-413d-988a-b83a113042e1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:37.857297 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf2c21-07de-413d-988a-b83a113042e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:37.857795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf2c21-07de-413d-988a-b83a113042e1 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:37.866091 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf2c21-07de-413d-988a-b83a113042e1 None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:37.868832 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf2c21-07de-413d-988a-b83a113042e1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:37.873156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 93/174] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:37 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:37.878034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf2c21-07de-413d-988a-b83a113042e1 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:37.967390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:37.968499 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:38.053453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21241971-9715-4bd8-9465-dc197c38d649 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:38.097530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b819a-a8bc-4e1c-bdd9-99da89debe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:38.098177 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b819a-a8bc-4e1c-bdd9-99da89debe07 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:38.098982 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b819a-a8bc-4e1c-bdd9-99da89debe07 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:38.099251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b819a-a8bc-4e1c-bdd9-99da89debe07 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:38.111405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b819a-a8bc-4e1c-bdd9-99da89debe07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:38.117564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-b96b819a-a8bc-4e1c-bdd9-99da89debe07 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:08:38.117564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:38.117564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:38.117564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:38.117564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:38.117564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:38.117564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:38.117564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:38.117564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:38.117564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:38.117564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:38.117564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:38.117564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:38.117564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:38.117564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:38.117564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:38.119295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:38.119295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:38.119295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:38.119295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:38.119295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:38.119295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:38.119295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:38.119295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:38.119295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:38.119295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:38.119295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:38.119295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:38.119295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:38.119295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:38.119295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:38.119295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:38.121788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:38.121788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:38.121788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:38.121788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:38.121788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:38.121788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:38.121788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:38.121788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:38.121788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:38.121788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:38.121788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:08:38.121788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:08:38.121788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 82/175] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:38.121788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b819a-a8bc-4e1c-bdd9-99da89debe07 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:38.147984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf2c21-07de-413d-988a-b83a113042e1 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:38.211938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5d7dd-79e7-4fd1-8ab9-e5af64fe9110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:38.213016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5d7dd-79e7-4fd1-8ab9-e5af64fe9110 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:38.213544 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5d7dd-79e7-4fd1-8ab9-e5af64fe9110 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:38.214030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5d7dd-79e7-4fd1-8ab9-e5af64fe9110 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:38.216126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5d7dd-79e7-4fd1-8ab9-e5af64fe9110 None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:38.218939 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5d7dd-79e7-4fd1-8ab9-e5af64fe9110 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:38.222576 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 94/176] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:37 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:38.224041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5d7dd-79e7-4fd1-8ab9-e5af64fe9110 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:38.296846 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b819a-a8bc-4e1c-bdd9-99da89debe07 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:38.352127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db6315-d25d-449a-a700-a2725de41d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:38.352770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db6315-d25d-449a-a700-a2725de41d7e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:38.353085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db6315-d25d-449a-a700-a2725de41d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:38.353373 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db6315-d25d-449a-a700-a2725de41d7e None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:38.354044 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db6315-d25d-449a-a700-a2725de41d7e None admin] RBAC: Adding query filter params (name=member) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:38.354383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db6315-d25d-449a-a700-a2725de41d7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:38.356078 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db6315-d25d-449a-a700-a2725de41d7e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:38.363727 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:38.376608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 83/177] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:08:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:38.379462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e1e8b-89e9-45e1-beac-26e106cac741 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:38.380293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e1e8b-89e9-45e1-beac-26e106cac741 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:38.380793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e1e8b-89e9-45e1-beac-26e106cac741 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:38.383708 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 84/178] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:38.384978 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e1e8b-89e9-45e1-beac-26e106cac741 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:38.393924 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:38.423057 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5d7dd-79e7-4fd1-8ab9-e5af64fe9110 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:38.470383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf936eea-0770-4278-9d76-a3cf3a0d41b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:38.471024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf936eea-0770-4278-9d76-a3cf3a0d41b0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:38.471317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf936eea-0770-4278-9d76-a3cf3a0d41b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:38.471597 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf936eea-0770-4278-9d76-a3cf3a0d41b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:38.472346 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf936eea-0770-4278-9d76-a3cf3a0d41b0 None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=709dbd6287a240f3b6044bf6d6e63334, user.id=a7b11924f137447a8c88ac81ee52e9ff) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:38.472668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf936eea-0770-4278-9d76-a3cf3a0d41b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:38.475230 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf936eea-0770-4278-9d76-a3cf3a0d41b0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:38.487780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 95/179] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:38 2022] GET /identity/v3/role_assignments?user.id=a7b11924f137447a8c88ac81ee52e9ff&scope.project.id=709dbd6287a240f3b6044bf6d6e63334&role.id=dc5d71ab685e420697f520aa3a368177 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:38.490284 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf936eea-0770-4278-9d76-a3cf3a0d41b0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:38.604949 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e1e8b-89e9-45e1-beac-26e106cac741 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:38.670166 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf56e9-78ef-4989-883b-7699ea0b602a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:38.671699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf56e9-78ef-4989-883b-7699ea0b602a None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:38.672526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf56e9-78ef-4989-883b-7699ea0b602a None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:38.674088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf56e9-78ef-4989-883b-7699ea0b602a None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:38.675612 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf56e9-78ef-4989-883b-7699ea0b602a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:38.676414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf56e9-78ef-4989-883b-7699ea0b602a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:38.680254 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf56e9-78ef-4989-883b-7699ea0b602a None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:38.698162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 85/180] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:08:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:38.700764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7b06e-8c0f-4eee-94b5-204cd5b3642f None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:38.701569 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7b06e-8c0f-4eee-94b5-204cd5b3642f None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:38.702239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7b06e-8c0f-4eee-94b5-204cd5b3642f None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:38.705657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 86/181] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:38.707149 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf936eea-0770-4278-9d76-a3cf3a0d41b0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:38.707557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7b06e-8c0f-4eee-94b5-204cd5b3642f None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:38.774266 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd33a6-e5f5-4ab8-92e5-4d8e2f09ac32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:38.776277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd33a6-e5f5-4ab8-92e5-4d8e2f09ac32 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:38.777120 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd33a6-e5f5-4ab8-92e5-4d8e2f09ac32 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:38.777735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd33a6-e5f5-4ab8-92e5-4d8e2f09ac32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:38.779030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd33a6-e5f5-4ab8-92e5-4d8e2f09ac32 None admin] RBAC: Adding query filter params (role.id=fad7ba55962b40459b40dcc2bad395b2, scope.project.id=709dbd6287a240f3b6044bf6d6e63334, user.id=2a83b38dedc44dd89eb854ab6be2e94d) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:38.779604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd33a6-e5f5-4ab8-92e5-4d8e2f09ac32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:38.782591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd33a6-e5f5-4ab8-92e5-4d8e2f09ac32 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:38.794546 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 96/182] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:38 2022] GET /identity/v3/role_assignments?user.id=2a83b38dedc44dd89eb854ab6be2e94d&scope.project.id=709dbd6287a240f3b6044bf6d6e63334&role.id=fad7ba55962b40459b40dcc2bad395b2 => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:38.796336 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba58624-e9ff-4d73-847d-bebfd48959c7 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:38.796860 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba58624-e9ff-4d73-847d-bebfd48959c7 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:38.797622 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba58624-e9ff-4d73-847d-bebfd48959c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:38.828920 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:38.839949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9ba58624-e9ff-4d73-847d-bebfd48959c7 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:38.841519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9ba58624-e9ff-4d73-847d-bebfd48959c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:38.855501 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:38.892560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7b06e-8c0f-4eee-94b5-204cd5b3642f None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:38.947598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fd3b4-83f0-406b-9fd0-709acc58755b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:38.948634 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fd3b4-83f0-406b-9fd0-709acc58755b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:38.949203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fd3b4-83f0-406b-9fd0-709acc58755b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:38.949735 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fd3b4-83f0-406b-9fd0-709acc58755b None admin] PATH_INFO: `/v3/users/2a83b38dedc44dd89eb854ab6be2e94d` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:38.952061 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72fd3b4-83f0-406b-9fd0-709acc58755b None admin] RBAC: Authorizing `identity:get_user(user_id=2a83b38dedc44dd89eb854ab6be2e94d)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:38.953984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72fd3b4-83f0-406b-9fd0-709acc58755b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:38.956491 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 87/183] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:38 2022] GET /identity/v3/users/2a83b38dedc44dd89eb854ab6be2e94d => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:38.958925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fd3b4-83f0-406b-9fd0-709acc58755b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:38.980392 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 97/184] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:38.982385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49712eb9-9906-447d-a708-c102f482091b None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:38.982948 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49712eb9-9906-447d-a708-c102f482091b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:38.983481 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49712eb9-9906-447d-a708-c102f482091b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:39.010808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-49712eb9-9906-447d-a708-c102f482091b None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:39.013383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-49712eb9-9906-447d-a708-c102f482091b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:39.108626 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 98/185] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:39.112752 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49712eb9-9906-447d-a708-c102f482091b None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:39.183486 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fd3b4-83f0-406b-9fd0-709acc58755b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:39.247697 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:39.256029 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bba501-9565-4972-9945-c369a9ed12d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:39.257404 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bba501-9565-4972-9945-c369a9ed12d5 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:39.258245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bba501-9565-4972-9945-c369a9ed12d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:39.258729 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bba501-9565-4972-9945-c369a9ed12d5 None admin] PATH_INFO: `/v3/users/ac799727c48b4492940172c9924e103f` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:39.261004 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bba501-9565-4972-9945-c369a9ed12d5 None admin] RBAC: Authorizing `identity:get_user(user_id=ac799727c48b4492940172c9924e103f)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:39.264042 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bba501-9565-4972-9945-c369a9ed12d5 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:39.267673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 88/186] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:38 2022] GET /identity/v3/users/ac799727c48b4492940172c9924e103f => generated 289 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:39.270118 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c050a26-e9be-4101-a264-65f1ef6b5f13 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:39.270625 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c050a26-e9be-4101-a264-65f1ef6b5f13 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:39.271122 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c050a26-e9be-4101-a264-65f1ef6b5f13 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:39.274318 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 89/187] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:39.276228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c050a26-e9be-4101-a264-65f1ef6b5f13 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:39.285410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:39.338086 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49712eb9-9906-447d-a708-c102f482091b None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:39.393522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05515820-6e36-44dc-8fd7-c27cad1c9e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:39.394577 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05515820-6e36-44dc-8fd7-c27cad1c9e35 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:39.395533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05515820-6e36-44dc-8fd7-c27cad1c9e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:39.395533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05515820-6e36-44dc-8fd7-c27cad1c9e35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:39.453244 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c050a26-e9be-4101-a264-65f1ef6b5f13 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:39.485723 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05515820-6e36-44dc-8fd7-c27cad1c9e35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:39.487250 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05515820-6e36-44dc-8fd7-c27cad1c9e35 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:39.502112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f4e4f-d594-4c78-ba03-da68c9effd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:39.503043 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f4e4f-d594-4c78-ba03-da68c9effd1b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:39.503455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f4e4f-d594-4c78-ba03-da68c9effd1b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:39.503872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f4e4f-d594-4c78-ba03-da68c9effd1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:39.586110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:39.596171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:39.602389 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f4e4f-d594-4c78-ba03-da68c9effd1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:39.604813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f4e4f-d594-4c78-ba03-da68c9effd1b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:39.636669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 99/188] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:08:39.638692 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb52561-c458-40db-bb5c-543ade976356 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:39.639144 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb52561-c458-40db-bb5c-543ade976356 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:39.639588 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb52561-c458-40db-bb5c-543ade976356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:39.666169 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6fb52561-c458-40db-bb5c-543ade976356 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:39.667376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6fb52561-c458-40db-bb5c-543ade976356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:39.747712 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 90/189] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 22:08:39.748594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f4e4f-d594-4c78-ba03-da68c9effd1b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:39.769577 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 100/190] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:39.770587 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb52561-c458-40db-bb5c-543ade976356 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:39.912914 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:39.918964 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f4e4f-d594-4c78-ba03-da68c9effd1b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:39.944348 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb52561-c458-40db-bb5c-543ade976356 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:39.951074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:39.967816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcaa82-c04b-490d-a155-2ee64c8311ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:39.968649 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcaa82-c04b-490d-a155-2ee64c8311ee None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:39.969046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcaa82-c04b-490d-a155-2ee64c8311ee None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:39.969492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcaa82-c04b-490d-a155-2ee64c8311ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:39.983805 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcaa82-c04b-490d-a155-2ee64c8311ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:39.989824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-43bcaa82-c04b-490d-a155-2ee64c8311ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:08:39.989824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:39.989824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:39.989824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:39.989824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:39.989824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:39.989824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:39.989824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:39.989824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:39.989824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:39.989824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:39.989824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:39.989824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:39.989824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:39.989824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:39.989824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:39.991583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:39.991583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:39.991583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:39.991583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:39.991583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:39.991583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:39.991583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:39.991583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:39.991583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:39.991583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:39.991583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:39.991583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:39.991583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:39.991583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:39.991583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:39.991583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:39.999115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:39.999115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:39.999115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:39.999115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:39.999115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:39.999115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:39.999115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:39.999115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:39.999115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:39.999115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:39.999115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:08:39.999115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:08:39.999115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 91/191] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:08:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:39.999115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcaa82-c04b-490d-a155-2ee64c8311ee None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:40.028545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3ae8e-ab83-4ba1-9bd9-a3d4b7b2342b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:40.029562 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3ae8e-ab83-4ba1-9bd9-a3d4b7b2342b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:40.030010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3ae8e-ab83-4ba1-9bd9-a3d4b7b2342b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:40.030482 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3ae8e-ab83-4ba1-9bd9-a3d4b7b2342b None admin] PATH_INFO: `/v3/projects/4e75717e9b354d478ba30ddbc5c5d1c1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:40.032379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3ae8e-ab83-4ba1-9bd9-a3d4b7b2342b None admin] RBAC: Authorizing `identity:get_project(project_id=4e75717e9b354d478ba30ddbc5c5d1c1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:40.034828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3ae8e-ab83-4ba1-9bd9-a3d4b7b2342b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:40.038895 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 101/192] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:39 2022] GET /identity/v3/projects/4e75717e9b354d478ba30ddbc5c5d1c1 => generated 312 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:40.040228 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3ae8e-ab83-4ba1-9bd9-a3d4b7b2342b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:40.192602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcaa82-c04b-490d-a155-2ee64c8311ee None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:40.202813 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3ae8e-ab83-4ba1-9bd9-a3d4b7b2342b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:40.233302 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fc9ed-a906-4711-933e-4862cb216d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:40.233969 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fc9ed-a906-4711-933e-4862cb216d37 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:40.234253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fc9ed-a906-4711-933e-4862cb216d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:40.234531 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fc9ed-a906-4711-933e-4862cb216d37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:40.248835 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fc9ed-a906-4711-933e-4862cb216d37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:40.251209 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acb71c-8e2b-4190-9223-c29a81d3c0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:40.251936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acb71c-8e2b-4190-9223-c29a81d3c0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:40.252361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acb71c-8e2b-4190-9223-c29a81d3c0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:40.252704 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acb71c-8e2b-4190-9223-c29a81d3c0c9 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:40.253987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-e75fc9ed-a906-4711-933e-4862cb216d37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:40.253987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:40.253987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:40.253987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:40.253987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:40.253987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:40.253987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:40.253987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:40.253987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:40.253987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:40.253987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:40.253987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:40.253987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:40.253987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:40.253987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:40.253987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:40.256214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:40.256214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:40.256214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:40.256214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:40.256214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:40.256214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:40.256214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:40.256214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:40.256214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:40.256214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:40.256214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:40.256214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:40.256214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:40.256214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:40.256214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:40.256214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:40.257796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:40.257796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:40.257796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:40.257796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:40.257796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:40.257796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:40.257796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:40.257796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:40.257796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:40.257796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:40.257796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:40.257796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:08:40.259646 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acb71c-8e2b-4190-9223-c29a81d3c0c9 None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:40.259795 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 92/193] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:08:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:40.259945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acb71c-8e2b-4190-9223-c29a81d3c0c9 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:40.260086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fc9ed-a906-4711-933e-4862cb216d37 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:40.260459 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 102/194] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:40 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:40.262145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acb71c-8e2b-4190-9223-c29a81d3c0c9 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:40.296248 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:40.311608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:40.382529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acb71c-8e2b-4190-9223-c29a81d3c0c9 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:40.417918 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fc9ed-a906-4711-933e-4862cb216d37 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:40.424633 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032985b-ab8a-4073-a193-a01efe64d7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:40.425288 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032985b-ab8a-4073-a193-a01efe64d7fc None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:40.425572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032985b-ab8a-4073-a193-a01efe64d7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:40.425849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032985b-ab8a-4073-a193-a01efe64d7fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:40.427213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e032985b-ab8a-4073-a193-a01efe64d7fc None admin] RBAC: Adding query filter params (role.id=fad7ba55962b40459b40dcc2bad395b2, scope.project.id=4e75717e9b354d478ba30ddbc5c5d1c1, user.id=2a83b38dedc44dd89eb854ab6be2e94d) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:40.427765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e032985b-ab8a-4073-a193-a01efe64d7fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:40.429475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e032985b-ab8a-4073-a193-a01efe64d7fc None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:40.435869 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 103/195] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:40 2022] GET /identity/v3/role_assignments?user.id=2a83b38dedc44dd89eb854ab6be2e94d&scope.project.id=4e75717e9b354d478ba30ddbc5c5d1c1&role.id=fad7ba55962b40459b40dcc2bad395b2 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:40.436716 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032985b-ab8a-4073-a193-a01efe64d7fc None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:40.485906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d4e05-fff0-411e-b2a6-945296a3c405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:40.486913 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d4e05-fff0-411e-b2a6-945296a3c405 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:40.487379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d4e05-fff0-411e-b2a6-945296a3c405 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:40.487866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d4e05-fff0-411e-b2a6-945296a3c405 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:40.489727 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d4e05-fff0-411e-b2a6-945296a3c405 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:40.490266 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d4e05-fff0-411e-b2a6-945296a3c405 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:40.492885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d4e05-fff0-411e-b2a6-945296a3c405 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:40.508158 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 93/196] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:08:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:40.509504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d4e05-fff0-411e-b2a6-945296a3c405 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:40.620584 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032985b-ab8a-4073-a193-a01efe64d7fc None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:40.624245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:40.636283 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:40.672723 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb13bb-bb84-46bd-9ac1-66d36ccf9fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:40.673663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb13bb-bb84-46bd-9ac1-66d36ccf9fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:40.674145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb13bb-bb84-46bd-9ac1-66d36ccf9fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:40.674421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb13bb-bb84-46bd-9ac1-66d36ccf9fb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:40.675651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb13bb-bb84-46bd-9ac1-66d36ccf9fb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:40.678011 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d4e05-fff0-411e-b2a6-945296a3c405 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:40.678147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb13bb-bb84-46bd-9ac1-66d36ccf9fb1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:40.680126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 104/197] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:40.680830 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb13bb-bb84-46bd-9ac1-66d36ccf9fb1 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:40.733639 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa62076-6884-4788-b76e-ca655d58589b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:40.734278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa62076-6884-4788-b76e-ca655d58589b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:40.734558 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa62076-6884-4788-b76e-ca655d58589b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:40.734830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa62076-6884-4788-b76e-ca655d58589b None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:40.735585 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa62076-6884-4788-b76e-ca655d58589b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:40.735897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa62076-6884-4788-b76e-ca655d58589b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:40.737460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa62076-6884-4788-b76e-ca655d58589b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:40.749477 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 94/198] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:40.750799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa62076-6884-4788-b76e-ca655d58589b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:40.843504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb13bb-bb84-46bd-9ac1-66d36ccf9fb1 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:40.910447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b64b94-97fe-473a-bbc5-6ac9c9be9900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:40.911882 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa62076-6884-4788-b76e-ca655d58589b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:40.912053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b64b94-97fe-473a-bbc5-6ac9c9be9900 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:40.912510 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b64b94-97fe-473a-bbc5-6ac9c9be9900 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:40.912995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b64b94-97fe-473a-bbc5-6ac9c9be9900 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:40.914419 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b64b94-97fe-473a-bbc5-6ac9c9be9900 None admin] RBAC: Adding query filter params (role.id=8d5b36826f914e958bd80fc21156e286, scope.project.id=709dbd6287a240f3b6044bf6d6e63334, user.id=ac799727c48b4492940172c9924e103f) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:40.914969 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b64b94-97fe-473a-bbc5-6ac9c9be9900 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:40.917735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b64b94-97fe-473a-bbc5-6ac9c9be9900 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:40.928700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 105/199] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:40 2022] GET /identity/v3/role_assignments?user.id=ac799727c48b4492940172c9924e103f&scope.project.id=709dbd6287a240f3b6044bf6d6e63334&role.id=8d5b36826f914e958bd80fc21156e286 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:40.930335 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b64b94-97fe-473a-bbc5-6ac9c9be9900 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:40.984557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52920c3a-60d1-402c-8545-7e6a4febee0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:40.986045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52920c3a-60d1-402c-8545-7e6a4febee0e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:40.986770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52920c3a-60d1-402c-8545-7e6a4febee0e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:40.986903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:40.987248 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52920c3a-60d1-402c-8545-7e6a4febee0e None admin] PATH_INFO: `/v3/users/130f9d15d298498e86402d953d50e95c` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:41.008346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:41.011800 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52920c3a-60d1-402c-8545-7e6a4febee0e None admin] RBAC: Authorizing `identity:get_user(user_id=130f9d15d298498e86402d953d50e95c)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:41.013795 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52920c3a-60d1-402c-8545-7e6a4febee0e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:41.019378 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 95/200] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:40 2022] GET /identity/v3/users/130f9d15d298498e86402d953d50e95c => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:41.020768 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52920c3a-60d1-402c-8545-7e6a4febee0e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:41.123002 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b64b94-97fe-473a-bbc5-6ac9c9be9900 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:41.191660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52920c3a-60d1-402c-8545-7e6a4febee0e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:41.195168 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1ac52-53f6-474b-b3a2-110728a59b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:41.195678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1ac52-53f6-474b-b3a2-110728a59b2c None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:41.197370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1ac52-53f6-474b-b3a2-110728a59b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:41.197370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1ac52-53f6-474b-b3a2-110728a59b2c None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:41.197673 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1ac52-53f6-474b-b3a2-110728a59b2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:41.199305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1ac52-53f6-474b-b3a2-110728a59b2c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:41.248338 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 106/201] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:08:40 2022] POST /identity/v3/users => generated 297 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:08:41.256837 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1ac52-53f6-474b-b3a2-110728a59b2c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:41.268390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b39a2-83b7-47b2-b15a-9ce53b217242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:41.269308 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b39a2-83b7-47b2-b15a-9ce53b217242 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:41.269707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b39a2-83b7-47b2-b15a-9ce53b217242 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:41.270159 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b39a2-83b7-47b2-b15a-9ce53b217242 None admin] PATH_INFO: `/v3/users/130f9d15d298498e86402d953d50e95c` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:41.272178 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b39a2-83b7-47b2-b15a-9ce53b217242 None admin] RBAC: Authorizing `identity:get_user(user_id=130f9d15d298498e86402d953d50e95c)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:41.277771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b39a2-83b7-47b2-b15a-9ce53b217242 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:41.277771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 96/202] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:41 2022] GET /identity/v3/users/130f9d15d298498e86402d953d50e95c => generated 283 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:41.286769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b39a2-83b7-47b2-b15a-9ce53b217242 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:41.408154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:41.437893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:41.455305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1ac52-53f6-474b-b3a2-110728a59b2c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:41.481762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b39a2-83b7-47b2-b15a-9ce53b217242 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:41.527046 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7632b90-aebc-45e4-896a-30251bca80f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:41.528501 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7632b90-aebc-45e4-896a-30251bca80f8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:41.528976 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7632b90-aebc-45e4-896a-30251bca80f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:41.529667 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7632b90-aebc-45e4-896a-30251bca80f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:41.554106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b14a0-7272-4047-b708-a823cb9361f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:41.556109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b14a0-7272-4047-b708-a823cb9361f6 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:41.556109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b14a0-7272-4047-b708-a823cb9361f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:41.556109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b14a0-7272-4047-b708-a823cb9361f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:41.678472 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7632b90-aebc-45e4-896a-30251bca80f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:41.681283 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7632b90-aebc-45e4-896a-30251bca80f8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:41.699464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b14a0-7272-4047-b708-a823cb9361f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:41.703548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b14a0-7272-4047-b708-a823cb9361f6 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:41.849282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:41.889122 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:41.891622 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 107/203] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:08:41.899420 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7632b90-aebc-45e4-896a-30251bca80f8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:41.913859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 97/204] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:08:41.919778 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b14a0-7272-4047-b708-a823cb9361f6 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:42.039264 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b14a0-7272-4047-b708-a823cb9361f6 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:42.051527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7632b90-aebc-45e4-896a-30251bca80f8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:42.080121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2566c2-a3b3-4be0-b896-9018348a3f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:42.080779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2566c2-a3b3-4be0-b896-9018348a3f50 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:42.081129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2566c2-a3b3-4be0-b896-9018348a3f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:42.081411 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2566c2-a3b3-4be0-b896-9018348a3f50 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:42.083279 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2566c2-a3b3-4be0-b896-9018348a3f50 None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:42.085047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2566c2-a3b3-4be0-b896-9018348a3f50 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:42.087075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 98/205] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:41 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:42.093189 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2566c2-a3b3-4be0-b896-9018348a3f50 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:42.095685 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0798c-128e-49ea-b167-d4fa1d3d5bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:42.096741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0798c-128e-49ea-b167-d4fa1d3d5bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:42.097665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0798c-128e-49ea-b167-d4fa1d3d5bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:42.098826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0798c-128e-49ea-b167-d4fa1d3d5bf8 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:42.100391 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0798c-128e-49ea-b167-d4fa1d3d5bf8 None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:42.102211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0798c-128e-49ea-b167-d4fa1d3d5bf8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:42.104307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 108/206] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:41 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:42.113620 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0798c-128e-49ea-b167-d4fa1d3d5bf8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:42.136565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:42.190507 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:42.244829 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0798c-128e-49ea-b167-d4fa1d3d5bf8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:42.249989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2566c2-a3b3-4be0-b896-9018348a3f50 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:42.295188 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af12732-d22d-4d39-ac82-c537246e6697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:42.296196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af12732-d22d-4d39-ac82-c537246e6697 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:42.296703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af12732-d22d-4d39-ac82-c537246e6697 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:42.297217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af12732-d22d-4d39-ac82-c537246e6697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:42.298809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af12732-d22d-4d39-ac82-c537246e6697 None admin] RBAC: Adding query filter params (role.id=3c32f171f34345e4958d31e3940092d6, scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a, user.id=130f9d15d298498e86402d953d50e95c) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:42.299374 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af12732-d22d-4d39-ac82-c537246e6697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:42.302246 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af12732-d22d-4d39-ac82-c537246e6697 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:42.310251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 109/207] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:42 2022] GET /identity/v3/role_assignments?user.id=130f9d15d298498e86402d953d50e95c&scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a&role.id=3c32f171f34345e4958d31e3940092d6 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:42.326274 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b508ce0-a8e7-4d24-b813-55f71c0efca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:42.327401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b508ce0-a8e7-4d24-b813-55f71c0efca4 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:42.328076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b508ce0-a8e7-4d24-b813-55f71c0efca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:42.328586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b508ce0-a8e7-4d24-b813-55f71c0efca4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:42.330137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b508ce0-a8e7-4d24-b813-55f71c0efca4 None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a, user.id=130f9d15d298498e86402d953d50e95c) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:42.330711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b508ce0-a8e7-4d24-b813-55f71c0efca4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:42.333948 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b508ce0-a8e7-4d24-b813-55f71c0efca4 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:42.345738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 99/208] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:42 2022] GET /identity/v3/role_assignments?user.id=130f9d15d298498e86402d953d50e95c&scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a&role.id=dc5d71ab685e420697f520aa3a368177 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:43.817896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de33626-9789-44c4-9b3d-3d806b8a3028 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:43.818894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de33626-9789-44c4-9b3d-3d806b8a3028 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:43.819386 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de33626-9789-44c4-9b3d-3d806b8a3028 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:43.821369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 110/209] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:43.828113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3492e-352e-44ea-8d51-a26dcb846006 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:43.828624 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3492e-352e-44ea-8d51-a26dcb846006 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:43.828936 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3492e-352e-44ea-8d51-a26dcb846006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:43.855806 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-f5f3492e-352e-44ea-8d51-a26dcb846006 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:43.857030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f5f3492e-352e-44ea-8d51-a26dcb846006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:43.941657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 100/210] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:44.230087 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de33626-9789-44c4-9b3d-3d806b8a3028 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:44.410175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de33626-9789-44c4-9b3d-3d806b8a3028 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:44.451333 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:44.456612 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cbce1-a36d-4f49-8527-a36217c24fa9 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:44.457261 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cbce1-a36d-4f49-8527-a36217c24fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:44.458315 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cbce1-a36d-4f49-8527-a36217c24fa9 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:44.459982 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 101/211] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:08:44.465987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ee1cf-0ba2-4f59-98a8-37e59feecc08 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:44.466594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ee1cf-0ba2-4f59-98a8-37e59feecc08 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:44.466913 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ee1cf-0ba2-4f59-98a8-37e59feecc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:44.471388 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029cb8e-ec08-4cec-b6eb-2adc087e82be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:44.472474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029cb8e-ec08-4cec-b6eb-2adc087e82be None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:44.473008 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029cb8e-ec08-4cec-b6eb-2adc087e82be None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:44.474160 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029cb8e-ec08-4cec-b6eb-2adc087e82be None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:44.495199 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029cb8e-ec08-4cec-b6eb-2adc087e82be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:44.498006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-ac8ee1cf-0ba2-4f59-98a8-37e59feecc08 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:44.499914 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ac8ee1cf-0ba2-4f59-98a8-37e59feecc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:44.506074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-d029cb8e-ec08-4cec-b6eb-2adc087e82be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:44.506074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:44.506074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:44.506074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:44.506074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:44.506074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:44.506074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:44.506074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:44.506074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:44.506074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:44.506074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:44.506074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:44.506074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:44.506074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:44.506074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:44.506074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:44.507013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:44.507013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:44.507013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:44.507013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:44.507013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:44.507013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:44.507013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:44.507013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:44.507013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:44.507013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:44.507013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:44.507013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:44.507013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:44.507013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:44.507013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:44.507013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:44.507792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:44.507792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:44.507792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:44.507792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:44.507792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:44.507792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:44.507792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:44.507792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:44.507792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:44.507792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:44.507792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:44.507792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:44.508401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 111/212] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:44.517568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029cb8e-ec08-4cec-b6eb-2adc087e82be None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:44.539474 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:44.615418 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 102/213] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:44.616616 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8bc6e-309a-484e-9174-7bb1e4aaf786 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:44.616977 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8bc6e-309a-484e-9174-7bb1e4aaf786 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:44.617285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8bc6e-309a-484e-9174-7bb1e4aaf786 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:44.618894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 103/214] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:44.627592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06d231-4b44-41e0-8119-70d810cccff6 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:44.628067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06d231-4b44-41e0-8119-70d810cccff6 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:44.628540 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06d231-4b44-41e0-8119-70d810cccff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:44.653852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-bb06d231-4b44-41e0-8119-70d810cccff6 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:44.655647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-bb06d231-4b44-41e0-8119-70d810cccff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:44.674262 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029cb8e-ec08-4cec-b6eb-2adc087e82be None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:44.721845 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a53f9a-651b-4d74-8bab-ffd7010bd0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:44.722466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a53f9a-651b-4d74-8bab-ffd7010bd0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:44.722742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a53f9a-651b-4d74-8bab-ffd7010bd0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:44.723534 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a53f9a-651b-4d74-8bab-ffd7010bd0b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:44.724214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a53f9a-651b-4d74-8bab-ffd7010bd0b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:44.724685 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a53f9a-651b-4d74-8bab-ffd7010bd0b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:44.727455 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a53f9a-651b-4d74-8bab-ffd7010bd0b6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:44.741104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 112/215] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:08:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:44.746938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a53f9a-651b-4d74-8bab-ffd7010bd0b6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:44.785844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 104/216] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:44.834429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f3d97-63f0-4208-9c49-ff969d625878 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:44.835414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f3d97-63f0-4208-9c49-ff969d625878 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:44.835414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f3d97-63f0-4208-9c49-ff969d625878 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:44.836668 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 105/217] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:44.840478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:44.846055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a0dce-7117-4cdc-a153-719c96a09de0 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:44.846276 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a0dce-7117-4cdc-a153-719c96a09de0 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:44.846564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a0dce-7117-4cdc-a153-719c96a09de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:44.869079 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-931a0dce-7117-4cdc-a153-719c96a09de0 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:44.870809 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-931a0dce-7117-4cdc-a153-719c96a09de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:44.919308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a53f9a-651b-4d74-8bab-ffd7010bd0b6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:44.969553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 106/218] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:08:44.969553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a0dce-7117-4cdc-a153-719c96a09de0 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:44.978646 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8157fa-1a94-4772-84d9-ce5c2c7fc49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:44.979294 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8157fa-1a94-4772-84d9-ce5c2c7fc49a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:44.979571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8157fa-1a94-4772-84d9-ce5c2c7fc49a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:44.980421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8157fa-1a94-4772-84d9-ce5c2c7fc49a None admin] PATH_INFO: `/v3/users/ebf4437ca15744af87860f17b78d8c96` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:44.999346 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8157fa-1a94-4772-84d9-ce5c2c7fc49a None admin] RBAC: Authorizing `identity:get_user(user_id=ebf4437ca15744af87860f17b78d8c96)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:45.000982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8157fa-1a94-4772-84d9-ce5c2c7fc49a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:45.003171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 113/219] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:44 2022] GET /identity/v3/users/ebf4437ca15744af87860f17b78d8c96 => generated 297 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:45.004029 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:45.010583 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8157fa-1a94-4772-84d9-ce5c2c7fc49a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:45.123468 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a0dce-7117-4cdc-a153-719c96a09de0 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:45.163492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3d511-1e27-4c30-baa6-b295ee31928f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:45.164129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3d511-1e27-4c30-baa6-b295ee31928f None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:45.164407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3d511-1e27-4c30-baa6-b295ee31928f None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:45.164675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3d511-1e27-4c30-baa6-b295ee31928f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:45.165908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3d511-1e27-4c30-baa6-b295ee31928f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:45.167369 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3d511-1e27-4c30-baa6-b295ee31928f None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:45.177439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 107/220] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:45.177439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3d511-1e27-4c30-baa6-b295ee31928f None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:45.194814 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8157fa-1a94-4772-84d9-ce5c2c7fc49a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:45.199371 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:45.242015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dfa93-d115-405c-8ead-33fc89856fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:45.242631 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dfa93-d115-405c-8ead-33fc89856fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:45.242908 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dfa93-d115-405c-8ead-33fc89856fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:45.243180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dfa93-d115-405c-8ead-33fc89856fa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:45.314641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:45.327205 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4dfa93-d115-405c-8ead-33fc89856fa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:45.328690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4dfa93-d115-405c-8ead-33fc89856fa2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:45.335854 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3d511-1e27-4c30-baa6-b295ee31928f None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:45.383752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6fc152-e4e0-440d-ab38-b7765d113765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:45.384500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fc152-e4e0-440d-ab38-b7765d113765 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:45.384843 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fc152-e4e0-440d-ab38-b7765d113765 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:45.385270 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fc152-e4e0-440d-ab38-b7765d113765 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:45.400575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6fc152-e4e0-440d-ab38-b7765d113765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:45.405623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-eb6fc152-e4e0-440d-ab38-b7765d113765 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:08:45.405623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:45.405623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:45.405623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:45.405623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:45.405623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:45.405623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:45.405623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:45.405623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:45.405623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:45.405623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:45.405623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:45.405623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:45.405623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:45.405623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:45.405623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:45.406853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:45.406853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:45.406853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:45.406853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:45.406853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:45.406853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:45.406853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:45.406853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:45.406853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:45.406853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:45.406853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:45.406853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:45.406853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:45.406853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:45.406853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:45.406853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:45.407621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:45.407621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:45.407621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:45.407621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:45.407621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:45.407621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:45.407621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:45.407621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:45.407621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:45.407621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:45.407621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:08:45.407621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:08:45.407621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 108/221] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:08:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:45.408359 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6fc152-e4e0-440d-ab38-b7765d113765 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:45.462079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 114/222] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 22:08:45.463586 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dfa93-d115-405c-8ead-33fc89856fa2 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:45.469378 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:45.549605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6fc152-e4e0-440d-ab38-b7765d113765 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:45.597509 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2bf53-3141-4280-bb7f-40d6ade202be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:45.598322 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2bf53-3141-4280-bb7f-40d6ade202be None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:45.599007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2bf53-3141-4280-bb7f-40d6ade202be None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:45.599415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2bf53-3141-4280-bb7f-40d6ade202be None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:45.600417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2bf53-3141-4280-bb7f-40d6ade202be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:45.602364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2bf53-3141-4280-bb7f-40d6ade202be None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:45.615727 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dfa93-d115-405c-8ead-33fc89856fa2 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:45.632362 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 109/223] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:08:45 2022] POST /identity/v3/users => generated 297 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:08:45.632958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2bf53-3141-4280-bb7f-40d6ade202be None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:45.637896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:45.691797 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592ac4-bcce-4fd7-b53d-ada61672921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:45.692850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592ac4-bcce-4fd7-b53d-ada61672921e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:45.693378 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592ac4-bcce-4fd7-b53d-ada61672921e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:45.693861 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592ac4-bcce-4fd7-b53d-ada61672921e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:45.715441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3592ac4-bcce-4fd7-b53d-ada61672921e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:45.726176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-d3592ac4-bcce-4fd7-b53d-ada61672921e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:45.726176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:45.726176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:45.726176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:45.726176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:45.726176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:45.726176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:45.726176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:45.726176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:45.726176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:45.726176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:45.726176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:45.726176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:45.726176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:45.726176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:45.726176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:45.727162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:45.727162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:45.727162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:45.727162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:45.727162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:45.727162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:45.727162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:45.727162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:45.727162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:45.727162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:45.727162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:45.727162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:45.727162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:45.727162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:45.727162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:45.727162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:45.728015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:45.728015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:45.728015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:45.728015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:45.728015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:45.728015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:45.728015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:45.728015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:45.728015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:45.728015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:45.728015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:45.728015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:45.728675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 115/224] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:08:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:45.730001 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592ac4-bcce-4fd7-b53d-ada61672921e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:45.793016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2bf53-3141-4280-bb7f-40d6ade202be None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:45.859510 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b103ec5-9b8d-429d-a140-11c288116192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:45.860430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b103ec5-9b8d-429d-a140-11c288116192 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:45.861482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b103ec5-9b8d-429d-a140-11c288116192 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:45.861482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b103ec5-9b8d-429d-a140-11c288116192 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:45.861840 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b103ec5-9b8d-429d-a140-11c288116192 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:45.862263 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b103ec5-9b8d-429d-a140-11c288116192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:45.864932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b103ec5-9b8d-429d-a140-11c288116192 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:45.866839 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:45.877693 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 110/225] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:08:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:45.878494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b103ec5-9b8d-429d-a140-11c288116192 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:45.941117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592ac4-bcce-4fd7-b53d-ada61672921e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:46.003595 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:46.008806 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5b180-96a2-4da4-994e-369767a5aef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:46.009495 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5b180-96a2-4da4-994e-369767a5aef4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:46.009780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5b180-96a2-4da4-994e-369767a5aef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:46.010056 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5b180-96a2-4da4-994e-369767a5aef4 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:46.011276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5b180-96a2-4da4-994e-369767a5aef4 None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:46.012771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5b180-96a2-4da4-994e-369767a5aef4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:46.014837 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 116/226] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:45 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:46.015359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5b180-96a2-4da4-994e-369767a5aef4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:46.086477 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b103ec5-9b8d-429d-a140-11c288116192 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:46.158417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da483de6-781f-42af-b730-c6d93a81a63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:46.160070 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da483de6-781f-42af-b730-c6d93a81a63e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:46.161894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da483de6-781f-42af-b730-c6d93a81a63e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:46.162641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da483de6-781f-42af-b730-c6d93a81a63e None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:46.163665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da483de6-781f-42af-b730-c6d93a81a63e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:46.164208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da483de6-781f-42af-b730-c6d93a81a63e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:46.165864 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da483de6-781f-42af-b730-c6d93a81a63e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:46.183045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 111/227] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:46.184987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da483de6-781f-42af-b730-c6d93a81a63e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:46.208007 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5b180-96a2-4da4-994e-369767a5aef4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:46.280932 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:46.292995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd23cf9-b66d-44b8-8bee-0c33f6b57c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:46.294196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd23cf9-b66d-44b8-8bee-0c33f6b57c65 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:46.294702 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd23cf9-b66d-44b8-8bee-0c33f6b57c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:46.295111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd23cf9-b66d-44b8-8bee-0c33f6b57c65 None admin] PATH_INFO: `/v3/users/130f9d15d298498e86402d953d50e95c` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:46.299601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd23cf9-b66d-44b8-8bee-0c33f6b57c65 None admin] RBAC: Authorizing `identity:get_user(user_id=130f9d15d298498e86402d953d50e95c)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:46.303908 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd23cf9-b66d-44b8-8bee-0c33f6b57c65 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:46.307172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 117/228] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:46 2022] GET /identity/v3/users/130f9d15d298498e86402d953d50e95c => generated 283 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:46.308809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd23cf9-b66d-44b8-8bee-0c33f6b57c65 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:46.423692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da483de6-781f-42af-b730-c6d93a81a63e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:46.482802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:46.500799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff16f2b3-59a5-4cae-94e9-a85260b26c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:46.502429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16f2b3-59a5-4cae-94e9-a85260b26c65 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:46.503206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16f2b3-59a5-4cae-94e9-a85260b26c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:46.503964 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16f2b3-59a5-4cae-94e9-a85260b26c65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:46.506372 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff16f2b3-59a5-4cae-94e9-a85260b26c65 None admin] RBAC: Adding query filter params (role.id=fad7ba55962b40459b40dcc2bad395b2, scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a, user.id=ebf4437ca15744af87860f17b78d8c96) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:46.507357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff16f2b3-59a5-4cae-94e9-a85260b26c65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:46.510605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff16f2b3-59a5-4cae-94e9-a85260b26c65 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:46.529135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 112/229] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:46 2022] GET /identity/v3/role_assignments?user.id=ebf4437ca15744af87860f17b78d8c96&scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a&role.id=fad7ba55962b40459b40dcc2bad395b2 => generated 259 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:46.529399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd23cf9-b66d-44b8-8bee-0c33f6b57c65 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:46.531370 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff16f2b3-59a5-4cae-94e9-a85260b26c65 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:46.609776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11abd2-10e2-4117-a278-b827d4f3e054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:46.610883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11abd2-10e2-4117-a278-b827d4f3e054 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:46.611389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11abd2-10e2-4117-a278-b827d4f3e054 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:46.611882 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11abd2-10e2-4117-a278-b827d4f3e054 None admin] PATH_INFO: `/v3/users/130f9d15d298498e86402d953d50e95c` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:46.623115 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11abd2-10e2-4117-a278-b827d4f3e054 None admin] RBAC: Authorizing `identity:get_user(user_id=130f9d15d298498e86402d953d50e95c)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:46.625950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11abd2-10e2-4117-a278-b827d4f3e054 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:46.630020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 118/230] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:46 2022] GET /identity/v3/users/130f9d15d298498e86402d953d50e95c => generated 283 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:46.638766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11abd2-10e2-4117-a278-b827d4f3e054 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:46.871244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:46.873980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff16f2b3-59a5-4cae-94e9-a85260b26c65 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:46.923072 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047823d-bcb1-4234-9ab0-11b3f8798bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:46.924392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047823d-bcb1-4234-9ab0-11b3f8798bec None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:46.925209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047823d-bcb1-4234-9ab0-11b3f8798bec None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:46.925692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047823d-bcb1-4234-9ab0-11b3f8798bec None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:46.928080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047823d-bcb1-4234-9ab0-11b3f8798bec None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:46.930820 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047823d-bcb1-4234-9ab0-11b3f8798bec None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:46.932871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 113/231] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:46 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:46.942231 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11abd2-10e2-4117-a278-b827d4f3e054 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:46.943822 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047823d-bcb1-4234-9ab0-11b3f8798bec None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:46.992754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0d375-159b-457a-aa9b-6dbfe65457a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:46.993464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0d375-159b-457a-aa9b-6dbfe65457a9 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:46.993770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0d375-159b-457a-aa9b-6dbfe65457a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:46.994150 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0d375-159b-457a-aa9b-6dbfe65457a9 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:46.998004 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0d375-159b-457a-aa9b-6dbfe65457a9 None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:46.999685 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0d375-159b-457a-aa9b-6dbfe65457a9 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:47.001525 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 119/232] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:46 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:47.012990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0d375-159b-457a-aa9b-6dbfe65457a9 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:47.061814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:47.179901 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047823d-bcb1-4234-9ab0-11b3f8798bec None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:47.282880 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0d375-159b-457a-aa9b-6dbfe65457a9 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:47.295255 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd2bb6-d915-4ddd-ac1a-049c4291aaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:47.296401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd2bb6-d915-4ddd-ac1a-049c4291aaf3 None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:47.296927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd2bb6-d915-4ddd-ac1a-049c4291aaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:47.297417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd2bb6-d915-4ddd-ac1a-049c4291aaf3 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a/users/130f9d15d298498e86402d953d50e95c/roles/3c32f171f34345e4958d31e3940092d6` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:47.303747 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd2bb6-d915-4ddd-ac1a-049c4291aaf3 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc2c956c8a3f4d0f832f927cdc74095a, user_id=130f9d15d298498e86402d953d50e95c, role_id=3c32f171f34345e4958d31e3940092d6)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:47.305641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd2bb6-d915-4ddd-ac1a-049c4291aaf3 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:47.325817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714ba03-0d1f-4c8c-ac34-5ed97ba99d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:47.326474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714ba03-0d1f-4c8c-ac34-5ed97ba99d7a None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:47.326761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714ba03-0d1f-4c8c-ac34-5ed97ba99d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:47.327178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714ba03-0d1f-4c8c-ac34-5ed97ba99d7a None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a/users/130f9d15d298498e86402d953d50e95c/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:47.330524 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714ba03-0d1f-4c8c-ac34-5ed97ba99d7a None admin] RBAC: Authorizing `identity:create_grant(project_id=dc2c956c8a3f4d0f832f927cdc74095a, user_id=130f9d15d298498e86402d953d50e95c, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:47.333028 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714ba03-0d1f-4c8c-ac34-5ed97ba99d7a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:47.337730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:47.339286 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 114/233] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:08:46 2022] PUT /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a/users/130f9d15d298498e86402d953d50e95c/roles/3c32f171f34345e4958d31e3940092d6 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:08:47.375985 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 120/234] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:08:47 2022] PUT /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a/users/130f9d15d298498e86402d953d50e95c/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:08:49.041734 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a78b2-5ca4-4945-a190-4ebc458d7890 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:49.043257 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a78b2-5ca4-4945-a190-4ebc458d7890 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:49.044081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a78b2-5ca4-4945-a190-4ebc458d7890 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:49.047171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 115/235] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:49.056379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26131b3-5f1a-4c21-8f00-e7148be12d58 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:49.057210 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26131b3-5f1a-4c21-8f00-e7148be12d58 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:49.057210 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26131b3-5f1a-4c21-8f00-e7148be12d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:49.094314 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f26131b3-5f1a-4c21-8f00-e7148be12d58 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:49.095601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f26131b3-5f1a-4c21-8f00-e7148be12d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:49.127069 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33a97b-c694-48a0-9b6e-77a555e2a50d None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:49.127355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33a97b-c694-48a0-9b6e-77a555e2a50d None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:49.127665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33a97b-c694-48a0-9b6e-77a555e2a50d None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:49.129376 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 116/236] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:08:49.147126 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb1168-fa54-4e74-8948-83f2db31a6d3 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:49.147613 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb1168-fa54-4e74-8948-83f2db31a6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:49.148086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb1168-fa54-4e74-8948-83f2db31a6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:49.198210 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-e2eb1168-fa54-4e74-8948-83f2db31a6d3 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:49.199447 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e2eb1168-fa54-4e74-8948-83f2db31a6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:49.216457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 121/237] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 23 22:08:49.389740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 117/238] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:49.777212 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26131b3-5f1a-4c21-8f00-e7148be12d58 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:49.831756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb1168-fa54-4e74-8948-83f2db31a6d3 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:50.000781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:50.030671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26131b3-5f1a-4c21-8f00-e7148be12d58 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:50.062771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:50.075403 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb1168-fa54-4e74-8948-83f2db31a6d3 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:50.076202 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52299b3-2666-481f-a539-e19fd625cc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:50.077211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52299b3-2666-481f-a539-e19fd625cc07 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:50.077655 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52299b3-2666-481f-a539-e19fd625cc07 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:50.078128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52299b3-2666-481f-a539-e19fd625cc07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:50.079662 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52299b3-2666-481f-a539-e19fd625cc07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:50.081385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52299b3-2666-481f-a539-e19fd625cc07 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:50.083430 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 122/239] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:50.085196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ba212-ca3c-44ea-924f-115773bb3501 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:50.085791 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ba212-ca3c-44ea-924f-115773bb3501 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:50.086320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ba212-ca3c-44ea-924f-115773bb3501 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:50.087921 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 123/240] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:50.093319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ba212-ca3c-44ea-924f-115773bb3501 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:50.118748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99275559-cccd-47a0-82fa-3d847e7fe97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:50.119816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99275559-cccd-47a0-82fa-3d847e7fe97e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:50.120312 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99275559-cccd-47a0-82fa-3d847e7fe97e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:50.120793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99275559-cccd-47a0-82fa-3d847e7fe97e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:50.143676 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99275559-cccd-47a0-82fa-3d847e7fe97e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:50.155871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-99275559-cccd-47a0-82fa-3d847e7fe97e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:08:50.155871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:50.155871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:50.155871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:50.155871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:50.155871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:50.155871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:50.155871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:50.155871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:50.155871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:50.155871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:50.155871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:50.155871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:50.155871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:50.155871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:50.155871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:50.156877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:50.156877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:50.156877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:50.156877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:50.156877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:50.156877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:50.156877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:50.156877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:50.156877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:50.156877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:50.156877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:50.156877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:50.156877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:50.156877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:50.156877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:50.156877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:50.157730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:50.157730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:50.157730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:50.157730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:50.157730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:50.157730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:50.157730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:50.157730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:50.157730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:50.157730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:50.157730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:08:50.157730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:08:50.165281 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 118/241] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:50.168043 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35ada4-f8a8-464c-80e1-3b61962c1beb None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:50.168565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35ada4-f8a8-464c-80e1-3b61962c1beb None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:50.169071 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35ada4-f8a8-464c-80e1-3b61962c1beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:50.204623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-ea35ada4-f8a8-464c-80e1-3b61962c1beb None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:50.207092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ea35ada4-f8a8-464c-80e1-3b61962c1beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:50.247270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ba212-ca3c-44ea-924f-115773bb3501 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:50.312069 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986041a2-d187-4ae0-abe8-55539cefef96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:50.313649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986041a2-d187-4ae0-abe8-55539cefef96 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:50.314161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986041a2-d187-4ae0-abe8-55539cefef96 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:50.314793 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986041a2-d187-4ae0-abe8-55539cefef96 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:50.316032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986041a2-d187-4ae0-abe8-55539cefef96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:50.318360 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986041a2-d187-4ae0-abe8-55539cefef96 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:50.327391 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 119/242] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:50.329405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35ada4-f8a8-464c-80e1-3b61962c1beb None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:50.345983 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 124/243] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:08:50 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:08:50.367166 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1b6aa-4002-4a88-bc2d-ad67a5407793 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:50.367921 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1b6aa-4002-4a88-bc2d-ad67a5407793 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:50.368611 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1b6aa-4002-4a88-bc2d-ad67a5407793 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:50.371555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 125/244] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:50.381124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83eead6-882e-45b0-a900-c72cbc8ff528 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:50.381981 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83eead6-882e-45b0-a900-c72cbc8ff528 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:50.382661 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83eead6-882e-45b0-a900-c72cbc8ff528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:50.416636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e83eead6-882e-45b0-a900-c72cbc8ff528 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:50.419943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e83eead6-882e-45b0-a900-c72cbc8ff528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:50.440207 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:50.519474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:50.533227 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35ada4-f8a8-464c-80e1-3b61962c1beb None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:50.577261 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 126/245] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:50.580100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee24ae6-b21a-4a2e-9fdf-35ff338ed216 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:50.580917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee24ae6-b21a-4a2e-9fdf-35ff338ed216 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:50.581457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee24ae6-b21a-4a2e-9fdf-35ff338ed216 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:50.585208 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 127/246] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:50.587444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d206d52-1d99-4d99-87c8-04d5a7496414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:50.588478 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d206d52-1d99-4d99-87c8-04d5a7496414 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:50.588983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d206d52-1d99-4d99-87c8-04d5a7496414 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:50.589487 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d206d52-1d99-4d99-87c8-04d5a7496414 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:50.590503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d206d52-1d99-4d99-87c8-04d5a7496414 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:50.591058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d206d52-1d99-4d99-87c8-04d5a7496414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:50.592988 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d206d52-1d99-4d99-87c8-04d5a7496414 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:50.594742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e1c8b-4f8f-494c-93bf-6d10ba136598 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:50.595235 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e1c8b-4f8f-494c-93bf-6d10ba136598 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:50.595708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e1c8b-4f8f-494c-93bf-6d10ba136598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:50.617757 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 120/247] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:08:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:50.618179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-604e1c8b-4f8f-494c-93bf-6d10ba136598 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:50.619721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-604e1c8b-4f8f-494c-93bf-6d10ba136598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:50.626539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d206d52-1d99-4d99-87c8-04d5a7496414 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:50.803383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 128/248] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:08:50.872475 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d206d52-1d99-4d99-87c8-04d5a7496414 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:50.875639 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:50.879303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e1c8b-4f8f-494c-93bf-6d10ba136598 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:50.952801 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb2292-1a61-4f9e-add4-bdb202fb4d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:50.953903 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb2292-1a61-4f9e-add4-bdb202fb4d56 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:50.954355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb2292-1a61-4f9e-add4-bdb202fb4d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:50.954835 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb2292-1a61-4f9e-add4-bdb202fb4d56 None admin] PATH_INFO: `/v3/users/5a0c2c9466964d0d8ca3686550dd5709` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:50.989705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb2292-1a61-4f9e-add4-bdb202fb4d56 None admin] RBAC: Authorizing `identity:get_user(user_id=5a0c2c9466964d0d8ca3686550dd5709)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:50.992124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb2292-1a61-4f9e-add4-bdb202fb4d56 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:50.994301 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 121/249] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:50 2022] GET /identity/v3/users/5a0c2c9466964d0d8ca3686550dd5709 => generated 297 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:50.995405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb2292-1a61-4f9e-add4-bdb202fb4d56 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:51.093187 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e1c8b-4f8f-494c-93bf-6d10ba136598 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:51.108095 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:51.149301 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb2292-1a61-4f9e-add4-bdb202fb4d56 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:51.150423 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42daf3-94c6-4d6a-bce7-60f00ba475e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:51.153202 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42daf3-94c6-4d6a-bce7-60f00ba475e0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:51.153845 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42daf3-94c6-4d6a-bce7-60f00ba475e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:51.154406 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42daf3-94c6-4d6a-bce7-60f00ba475e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:51.179573 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42daf3-94c6-4d6a-bce7-60f00ba475e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:51.184763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-6b42daf3-94c6-4d6a-bce7-60f00ba475e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:51.184763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:51.184763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:51.184763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:51.184763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:51.184763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:51.184763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:51.184763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:51.184763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:51.184763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:51.184763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:51.184763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:51.184763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:51.184763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:51.184763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:51.184763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:51.185849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:51.185849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:51.185849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:51.185849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:51.185849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:51.185849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:51.185849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:51.185849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:51.185849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:51.185849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:51.185849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:51.185849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:51.185849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:51.185849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:51.185849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:51.185849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:51.186674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:51.186674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:51.186674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:51.186674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:51.186674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:51.186674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:51.186674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:51.186674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:51.186674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:51.186674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:51.186674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:51.186674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:51.186674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 129/250] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:51.187347 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42daf3-94c6-4d6a-bce7-60f00ba475e0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:51.215775 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f47d63-0c0e-480e-8aff-a98bb4003981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:51.216438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f47d63-0c0e-480e-8aff-a98bb4003981 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:51.216721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f47d63-0c0e-480e-8aff-a98bb4003981 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:51.216996 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f47d63-0c0e-480e-8aff-a98bb4003981 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:51.232396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f47d63-0c0e-480e-8aff-a98bb4003981 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:51.237589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-11f47d63-0c0e-480e-8aff-a98bb4003981 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:51.237589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:51.237589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:51.237589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:51.237589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:51.237589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:51.237589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:51.237589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:51.237589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:51.237589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:51.237589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:51.237589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:51.237589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:51.237589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:51.237589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:51.237589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:51.239109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:51.239109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:51.239109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:51.239109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:51.239109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:51.239109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:51.239109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:51.239109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:51.239109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:51.239109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:51.239109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:51.239109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:51.239109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:51.239109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:51.239109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:51.239109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:51.240026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:51.240026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:51.240026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:51.240026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:51.240026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:51.240026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:51.240026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:51.240026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:51.240026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:51.240026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:51.240026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:51.240026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:08:51.240026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 122/251] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:08:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:51.241388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f47d63-0c0e-480e-8aff-a98bb4003981 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:51.276056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:51.396006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42daf3-94c6-4d6a-bce7-60f00ba475e0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:51.438903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c857cc6-66ea-4bab-9158-0c83cec5bb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:51.439430 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c857cc6-66ea-4bab-9158-0c83cec5bb90 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:51.439699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c857cc6-66ea-4bab-9158-0c83cec5bb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:51.439956 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c857cc6-66ea-4bab-9158-0c83cec5bb90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:51.455150 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f47d63-0c0e-480e-8aff-a98bb4003981 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:51.510081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:51.515494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba785d-11c2-4243-8b22-aad9aad0caf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:51.516533 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba785d-11c2-4243-8b22-aad9aad0caf9 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:51.517202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba785d-11c2-4243-8b22-aad9aad0caf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:51.517701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba785d-11c2-4243-8b22-aad9aad0caf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:51.519453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba785d-11c2-4243-8b22-aad9aad0caf9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:51.521134 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba785d-11c2-4243-8b22-aad9aad0caf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:51.522880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba785d-11c2-4243-8b22-aad9aad0caf9 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:51.541822 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 123/252] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:08:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:51.546090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba785d-11c2-4243-8b22-aad9aad0caf9 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:51.567480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c857cc6-66ea-4bab-9158-0c83cec5bb90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:51.569691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c857cc6-66ea-4bab-9158-0c83cec5bb90 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:51.689944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:51.717310 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba785d-11c2-4243-8b22-aad9aad0caf9 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:51.760659 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 130/253] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:08:51.765301 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c857cc6-66ea-4bab-9158-0c83cec5bb90 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:51.809692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2164ed-d459-4803-ae06-968b1ddc6244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:51.810540 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2164ed-d459-4803-ae06-968b1ddc6244 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:51.811088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2164ed-d459-4803-ae06-968b1ddc6244 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:51.811551 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2164ed-d459-4803-ae06-968b1ddc6244 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:51.812541 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2164ed-d459-4803-ae06-968b1ddc6244 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:51.813093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2164ed-d459-4803-ae06-968b1ddc6244 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:51.814718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2164ed-d459-4803-ae06-968b1ddc6244 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:51.826423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 124/254] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:51.827659 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2164ed-d459-4803-ae06-968b1ddc6244 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:51.874336 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:51.955174 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c857cc6-66ea-4bab-9158-0c83cec5bb90 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:52.017804 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2164ed-d459-4803-ae06-968b1ddc6244 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:52.031966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e305a7b-9123-4f43-beb1-356a10a2413d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:52.032990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e305a7b-9123-4f43-beb1-356a10a2413d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:52.033508 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e305a7b-9123-4f43-beb1-356a10a2413d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:52.033994 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e305a7b-9123-4f43-beb1-356a10a2413d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:52.046674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e305a7b-9123-4f43-beb1-356a10a2413d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:52.053003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-2e305a7b-9123-4f43-beb1-356a10a2413d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:08:52.053003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:52.053003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:52.053003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:52.053003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:52.053003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:52.053003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:52.053003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:52.053003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:52.053003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:52.053003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:52.053003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:52.053003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:52.053003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:52.053003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:52.053003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:52.053936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:52.053936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:52.053936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:52.053936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:52.053936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:52.053936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:52.053936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:52.053936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:52.053936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:52.053936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:52.053936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:52.053936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:52.053936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:52.053936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:52.053936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:52.053936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:52.054713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:52.054713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:52.054713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:52.054713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:52.054713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:52.054713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:52.054713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:52.054713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:52.054713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:52.054713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:52.054713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:08:52.054713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:52.057087 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 131/255] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:08:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:52.058148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e305a7b-9123-4f43-beb1-356a10a2413d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:52.100762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54264b30-0e7b-45c6-8dee-309334eac69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:52.101903 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54264b30-0e7b-45c6-8dee-309334eac69c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:52.102739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54264b30-0e7b-45c6-8dee-309334eac69c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:52.103221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54264b30-0e7b-45c6-8dee-309334eac69c None admin] PATH_INFO: `/v3/users/ebf4437ca15744af87860f17b78d8c96` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:52.106368 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54264b30-0e7b-45c6-8dee-309334eac69c None admin] RBAC: Authorizing `identity:get_user(user_id=ebf4437ca15744af87860f17b78d8c96)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:52.109111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54264b30-0e7b-45c6-8dee-309334eac69c None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:52.114091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 125/256] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:51 2022] GET /identity/v3/users/ebf4437ca15744af87860f17b78d8c96 => generated 297 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:52.120930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54264b30-0e7b-45c6-8dee-309334eac69c None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:52.163124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:52.258701 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e305a7b-9123-4f43-beb1-356a10a2413d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:52.305287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:52.314226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7caa2e-0d46-498c-8328-8a8f255ca045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:52.315323 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7caa2e-0d46-498c-8328-8a8f255ca045 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:52.316080 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7caa2e-0d46-498c-8328-8a8f255ca045 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:52.317258 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7caa2e-0d46-498c-8328-8a8f255ca045 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:52.318851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7caa2e-0d46-498c-8328-8a8f255ca045 None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:52.321099 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7caa2e-0d46-498c-8328-8a8f255ca045 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:52.321274 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54264b30-0e7b-45c6-8dee-309334eac69c None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:52.323411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 132/257] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:52 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:52.324733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7caa2e-0d46-498c-8328-8a8f255ca045 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:52.376441 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9009b3-6c2b-4a2a-ad6d-82afe871415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:52.377121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9009b3-6c2b-4a2a-ad6d-82afe871415e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:52.377409 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9009b3-6c2b-4a2a-ad6d-82afe871415e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:52.377684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9009b3-6c2b-4a2a-ad6d-82afe871415e None admin] PATH_INFO: `/v3/users/130f9d15d298498e86402d953d50e95c` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:52.380078 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9009b3-6c2b-4a2a-ad6d-82afe871415e None admin] RBAC: Authorizing `identity:get_user(user_id=130f9d15d298498e86402d953d50e95c)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:52.382351 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9009b3-6c2b-4a2a-ad6d-82afe871415e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:52.391561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 126/258] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:52 2022] GET /identity/v3/users/130f9d15d298498e86402d953d50e95c => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:52.391561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9009b3-6c2b-4a2a-ad6d-82afe871415e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:52.483771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7caa2e-0d46-498c-8328-8a8f255ca045 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:52.493926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:52.525879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9009b3-6c2b-4a2a-ad6d-82afe871415e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:52.533385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90363293-51ad-40e6-aead-8b57e0c2dfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:52.534415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90363293-51ad-40e6-aead-8b57e0c2dfb7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:52.534905 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90363293-51ad-40e6-aead-8b57e0c2dfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:52.536044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90363293-51ad-40e6-aead-8b57e0c2dfb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:52.537258 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90363293-51ad-40e6-aead-8b57e0c2dfb7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:52.537809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90363293-51ad-40e6-aead-8b57e0c2dfb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:52.540082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90363293-51ad-40e6-aead-8b57e0c2dfb7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:52.559490 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 133/259] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:08:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:52.560774 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90363293-51ad-40e6-aead-8b57e0c2dfb7 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:52.586745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792b359-24c4-43f7-9353-3928ec16ace1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:52.587841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b359-24c4-43f7-9353-3928ec16ace1 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:52.588414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b359-24c4-43f7-9353-3928ec16ace1 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:52.588909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b359-24c4-43f7-9353-3928ec16ace1 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:52.590712 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792b359-24c4-43f7-9353-3928ec16ace1 None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:52.592775 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792b359-24c4-43f7-9353-3928ec16ace1 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:52.595015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 127/260] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:52 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:52.596426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792b359-24c4-43f7-9353-3928ec16ace1 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:52.682403 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:52.740536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792b359-24c4-43f7-9353-3928ec16ace1 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:52.754758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90363293-51ad-40e6-aead-8b57e0c2dfb7 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:52.785467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85aa31d-38a2-4586-99ae-f780ba5e38ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:52.786799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85aa31d-38a2-4586-99ae-f780ba5e38ec None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:52.787310 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85aa31d-38a2-4586-99ae-f780ba5e38ec None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:52.787877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85aa31d-38a2-4586-99ae-f780ba5e38ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:52.796373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0852b73-2616-481b-a3cb-a97ff9614940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:52.797005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0852b73-2616-481b-a3cb-a97ff9614940 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:52.797322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0852b73-2616-481b-a3cb-a97ff9614940 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:52.797599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0852b73-2616-481b-a3cb-a97ff9614940 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:52.798365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0852b73-2616-481b-a3cb-a97ff9614940 None admin] RBAC: Adding query filter params (role.id=8d5b36826f914e958bd80fc21156e286, scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a, user.id=5a0c2c9466964d0d8ca3686550dd5709) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:52.798675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0852b73-2616-481b-a3cb-a97ff9614940 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:52.800207 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0852b73-2616-481b-a3cb-a97ff9614940 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:52.807564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 134/261] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:52 2022] GET /identity/v3/role_assignments?user.id=5a0c2c9466964d0d8ca3686550dd5709&scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a&role.id=8d5b36826f914e958bd80fc21156e286 => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:52.808478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0852b73-2616-481b-a3cb-a97ff9614940 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:52.816010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:52.893819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85aa31d-38a2-4586-99ae-f780ba5e38ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:52.897764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85aa31d-38a2-4586-99ae-f780ba5e38ec None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:53.079858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0852b73-2616-481b-a3cb-a97ff9614940 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:53.156405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:53.185365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf060a-07f6-4afd-8a9e-04de5f9b717c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:53.187035 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf060a-07f6-4afd-8a9e-04de5f9b717c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:53.187907 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf060a-07f6-4afd-8a9e-04de5f9b717c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:53.189706 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf060a-07f6-4afd-8a9e-04de5f9b717c None admin] PATH_INFO: `/v3/users/130f9d15d298498e86402d953d50e95c` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:53.192861 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf060a-07f6-4afd-8a9e-04de5f9b717c None admin] RBAC: Authorizing `identity:get_user(user_id=130f9d15d298498e86402d953d50e95c)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:53.198116 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf060a-07f6-4afd-8a9e-04de5f9b717c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:53.201422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 135/262] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:52 2022] GET /identity/v3/users/130f9d15d298498e86402d953d50e95c => generated 283 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:53.203062 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf060a-07f6-4afd-8a9e-04de5f9b717c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:53.204441 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 128/263] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:08:53.209193 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85aa31d-38a2-4586-99ae-f780ba5e38ec None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:53.329308 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:53.375384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf060a-07f6-4afd-8a9e-04de5f9b717c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:53.417565 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2018e21-aeb4-4b47-91a2-f6699469e885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:53.419221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2018e21-aeb4-4b47-91a2-f6699469e885 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:53.420016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2018e21-aeb4-4b47-91a2-f6699469e885 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:53.420856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2018e21-aeb4-4b47-91a2-f6699469e885 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a/users/ebf4437ca15744af87860f17b78d8c96/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:53.425365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2018e21-aeb4-4b47-91a2-f6699469e885 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc2c956c8a3f4d0f832f927cdc74095a, user_id=ebf4437ca15744af87860f17b78d8c96, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:53.429014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2018e21-aeb4-4b47-91a2-f6699469e885 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:53.434367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85aa31d-38a2-4586-99ae-f780ba5e38ec None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:53.457589 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 136/264] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:08:53 2022] PUT /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a/users/ebf4437ca15744af87860f17b78d8c96/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:08:53.459279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2018e21-aeb4-4b47-91a2-f6699469e885 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:53.513152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3576e0c-420f-4308-a06a-45dd24f46c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:53.513152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3576e0c-420f-4308-a06a-45dd24f46c15 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:53.513152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3576e0c-420f-4308-a06a-45dd24f46c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:53.513152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3576e0c-420f-4308-a06a-45dd24f46c15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:53.569369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:53.707739 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2018e21-aeb4-4b47-91a2-f6699469e885 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:53.718235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3576e0c-420f-4308-a06a-45dd24f46c15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:53.721617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3576e0c-420f-4308-a06a-45dd24f46c15 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:53.755595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65217e6-acfc-4a70-8ef1-49e4d05c6a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:53.756251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65217e6-acfc-4a70-8ef1-49e4d05c6a96 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:53.756530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65217e6-acfc-4a70-8ef1-49e4d05c6a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:53.756802 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65217e6-acfc-4a70-8ef1-49e4d05c6a96 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:53.758079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65217e6-acfc-4a70-8ef1-49e4d05c6a96 None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:53.759574 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65217e6-acfc-4a70-8ef1-49e4d05c6a96 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:53.766835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 137/265] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:53 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:53.766835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1442225-2785-4fd3-8888-efa86fec641f None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:53.766835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1442225-2785-4fd3-8888-efa86fec641f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:53.766835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1442225-2785-4fd3-8888-efa86fec641f None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:53.766835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 138/266] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:53.768193 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1442225-2785-4fd3-8888-efa86fec641f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:53.801596 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:53.871254 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 129/267] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 22:08:53.873322 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd8245-24df-493e-8c86-575c8f54d791 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:53.873834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd8245-24df-493e-8c86-575c8f54d791 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:53.874310 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd8245-24df-493e-8c86-575c8f54d791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:53.896850 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-68fd8245-24df-493e-8c86-575c8f54d791 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:53.898491 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-68fd8245-24df-493e-8c86-575c8f54d791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:53.936882 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1442225-2785-4fd3-8888-efa86fec641f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:53.987544 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:54.007645 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 130/268] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:54.010918 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90434cc-bbef-4836-80b3-b820619adf48 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:54.011961 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90434cc-bbef-4836-80b3-b820619adf48 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:54.012750 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90434cc-bbef-4836-80b3-b820619adf48 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:54.015312 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a44f0e-aa06-4161-b397-aabfec44609b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:54.016239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a44f0e-aa06-4161-b397-aabfec44609b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:54.016556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 131/269] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:54.016890 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a44f0e-aa06-4161-b397-aabfec44609b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:54.017470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a44f0e-aa06-4161-b397-aabfec44609b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:54.018862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a44f0e-aa06-4161-b397-aabfec44609b None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a, user.id=130f9d15d298498e86402d953d50e95c) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:54.019976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90434cc-bbef-4836-80b3-b820619adf48 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:54.020142 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a44f0e-aa06-4161-b397-aabfec44609b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:54.023438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a44f0e-aa06-4161-b397-aabfec44609b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:54.044647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 139/270] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:53 2022] GET /identity/v3/role_assignments?user.id=130f9d15d298498e86402d953d50e95c&scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a&role.id=dc5d71ab685e420697f520aa3a368177 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:54.044647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3eea2c-c82b-42a0-9a65-8e75474c57c9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:54.044647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3eea2c-c82b-42a0-9a65-8e75474c57c9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:54.044647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3eea2c-c82b-42a0-9a65-8e75474c57c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:54.077245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-dc3eea2c-c82b-42a0-9a65-8e75474c57c9 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:54.079151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-dc3eea2c-c82b-42a0-9a65-8e75474c57c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:54.205108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:54.251924 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90434cc-bbef-4836-80b3-b820619adf48 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:54.280257 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 140/271] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:54.346058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64605cd-d12e-4568-986b-c01234ef4498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:54.348051 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64605cd-d12e-4568-986b-c01234ef4498 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:54.348594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64605cd-d12e-4568-986b-c01234ef4498 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:54.349178 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64605cd-d12e-4568-986b-c01234ef4498 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:54.351154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64605cd-d12e-4568-986b-c01234ef4498 None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:54.353735 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64605cd-d12e-4568-986b-c01234ef4498 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:54.356666 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 132/272] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:54 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:54.367748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3eea2c-c82b-42a0-9a65-8e75474c57c9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:54.563678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3eea2c-c82b-42a0-9a65-8e75474c57c9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:54.583307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:54.615806 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5e90f-9491-4b86-97df-a0841e2db129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:54.616828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5e90f-9491-4b86-97df-a0841e2db129 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:54.617362 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5e90f-9491-4b86-97df-a0841e2db129 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:54.617860 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5e90f-9491-4b86-97df-a0841e2db129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:54.619106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5e90f-9491-4b86-97df-a0841e2db129 None admin] RBAC: Adding query filter params (role.id=3c32f171f34345e4958d31e3940092d6, scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a, user.id=130f9d15d298498e86402d953d50e95c) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:54.619424 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5e90f-9491-4b86-97df-a0841e2db129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:54.621988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5e90f-9491-4b86-97df-a0841e2db129 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:54.637360 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 141/273] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:54 2022] GET /identity/v3/role_assignments?user.id=130f9d15d298498e86402d953d50e95c&scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a&role.id=3c32f171f34345e4958d31e3940092d6 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:54.760532 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64605cd-d12e-4568-986b-c01234ef4498 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:54.927050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5e90f-9491-4b86-97df-a0841e2db129 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:55.006792 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64605cd-d12e-4568-986b-c01234ef4498 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:55.084972 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5ef38-bd94-4698-b750-65d99ae6b146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:55.085688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5ef38-bd94-4698-b750-65d99ae6b146 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:55.086123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5ef38-bd94-4698-b750-65d99ae6b146 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:55.086576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5ef38-bd94-4698-b750-65d99ae6b146 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:55.105776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff5ef38-bd94-4698-b750-65d99ae6b146 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:55.111812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-fff5ef38-bd94-4698-b750-65d99ae6b146 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:55.111812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:55.111812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:55.111812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:55.111812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:55.111812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:55.111812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:55.111812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:55.111812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:55.111812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:55.111812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:55.111812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:55.111812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:55.111812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:55.111812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:55.111812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:55.112756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:55.112756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:55.112756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:55.112756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:55.112756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:55.112756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:55.112756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:55.112756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:55.112756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:55.112756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:55.112756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:55.112756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:55.112756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:55.112756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:55.112756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:55.112756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:55.114144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:55.114144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:55.114144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:55.114144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:55.114144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:55.114144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:55.114144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:55.114144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:55.114144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:55.114144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:55.114144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:55.114144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:08:55.117232 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 133/274] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:55.126467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5ef38-bd94-4698-b750-65d99ae6b146 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:55.157397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:55.169619 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5e90f-9491-4b86-97df-a0841e2db129 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:55.234628 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65cc94-d954-4577-ab1c-ec22cfa72e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:55.235263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65cc94-d954-4577-ab1c-ec22cfa72e65 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:55.235702 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65cc94-d954-4577-ab1c-ec22cfa72e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:55.236030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65cc94-d954-4577-ab1c-ec22cfa72e65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:55.237358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65cc94-d954-4577-ab1c-ec22cfa72e65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:55.240037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65cc94-d954-4577-ab1c-ec22cfa72e65 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:55.243636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 142/275] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:55.259819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65cc94-d954-4577-ab1c-ec22cfa72e65 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:55.331398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:55.367947 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5ef38-bd94-4698-b750-65d99ae6b146 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:55.423558 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b1647-559e-4a7a-8a50-e0a348319b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:55.424309 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b1647-559e-4a7a-8a50-e0a348319b0b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:55.424813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b1647-559e-4a7a-8a50-e0a348319b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:55.425359 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b1647-559e-4a7a-8a50-e0a348319b0b None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:55.427624 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b1647-559e-4a7a-8a50-e0a348319b0b None admin] RBAC: Adding query filter params (name=member) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:55.428195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b1647-559e-4a7a-8a50-e0a348319b0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:55.430948 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b1647-559e-4a7a-8a50-e0a348319b0b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:55.442292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65cc94-d954-4577-ab1c-ec22cfa72e65 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:55.449093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 134/276] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:08:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:55.454950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b1647-559e-4a7a-8a50-e0a348319b0b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:55.520017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47196d3e-c65b-45ac-afcb-2da35cbe2d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:55.521491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47196d3e-c65b-45ac-afcb-2da35cbe2d2e None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:55.522212 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47196d3e-c65b-45ac-afcb-2da35cbe2d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:55.522898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47196d3e-c65b-45ac-afcb-2da35cbe2d2e None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:55.524481 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47196d3e-c65b-45ac-afcb-2da35cbe2d2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:55.527833 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47196d3e-c65b-45ac-afcb-2da35cbe2d2e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:55.540529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:55.578303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 143/277] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:08:55 2022] POST /identity/v3/users => generated 293 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:08:55.615409 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b1647-559e-4a7a-8a50-e0a348319b0b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:55.693415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94ade7-d6fb-4e7b-a450-11380ebe9f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:55.695044 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94ade7-d6fb-4e7b-a450-11380ebe9f0c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:55.695819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94ade7-d6fb-4e7b-a450-11380ebe9f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:55.696572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94ade7-d6fb-4e7b-a450-11380ebe9f0c None admin] PATH_INFO: `/v3/users/4ed7d09ca57f4bb6a2f19e748b430dd4` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:55.727160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94ade7-d6fb-4e7b-a450-11380ebe9f0c None admin] RBAC: Authorizing `identity:get_user(user_id=4ed7d09ca57f4bb6a2f19e748b430dd4)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:55.730653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94ade7-d6fb-4e7b-a450-11380ebe9f0c None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:55.734287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 135/278] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:55 2022] GET /identity/v3/users/4ed7d09ca57f4bb6a2f19e748b430dd4 => generated 293 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:55.742242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47196d3e-c65b-45ac-afcb-2da35cbe2d2e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:55.907603 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47196d3e-c65b-45ac-afcb-2da35cbe2d2e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:55.911005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:55.971694 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabfcb2-1aeb-4fdc-b4b7-a158d313e2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:55.971694 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabfcb2-1aeb-4fdc-b4b7-a158d313e2d9 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:55.971694 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabfcb2-1aeb-4fdc-b4b7-a158d313e2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:55.971694 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabfcb2-1aeb-4fdc-b4b7-a158d313e2d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:55.973200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adabfcb2-1aeb-4fdc-b4b7-a158d313e2d9 None admin] RBAC: Adding query filter params (role.id=fad7ba55962b40459b40dcc2bad395b2, user.id=4ed7d09ca57f4bb6a2f19e748b430dd4, scope.system=all) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:55.973200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adabfcb2-1aeb-4fdc-b4b7-a158d313e2d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:55.974085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adabfcb2-1aeb-4fdc-b4b7-a158d313e2d9 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:55.981569 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 144/279] 10.222.0.88 () {56 vars in 1308 bytes} [Tue Aug 23 22:08:55 2022] GET /identity/v3/role_assignments?user.id=4ed7d09ca57f4bb6a2f19e748b430dd4&scope.system=all&role.id=fad7ba55962b40459b40dcc2bad395b2 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:56.021718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec87bd4-9f98-4c96-a439-b740b48d8fe1 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:56.022059 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec87bd4-9f98-4c96-a439-b740b48d8fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:56.022274 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec87bd4-9f98-4c96-a439-b740b48d8fe1 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:56.023686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 136/280] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:56.029698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dae67-6a99-415f-921f-0adc3f1a5386 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:56.030225 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dae67-6a99-415f-921f-0adc3f1a5386 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:56.030655 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dae67-6a99-415f-921f-0adc3f1a5386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:56.055694 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d72dae67-6a99-415f-921f-0adc3f1a5386 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:56.057305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d72dae67-6a99-415f-921f-0adc3f1a5386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:56.150726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 145/281] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:56.596909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea33eba-52a1-4950-8c05-3c28b72620b4 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:56.597987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea33eba-52a1-4950-8c05-3c28b72620b4 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:56.598064 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea33eba-52a1-4950-8c05-3c28b72620b4 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:56.599907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 137/282] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:08:56.605921 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7a5b7-55ac-457c-95c7-da632d0c701e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:56.606430 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7a5b7-55ac-457c-95c7-da632d0c701e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:56.606505 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7a5b7-55ac-457c-95c7-da632d0c701e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:56.638259 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-13e7a5b7-55ac-457c-95c7-da632d0c701e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:56.639485 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-13e7a5b7-55ac-457c-95c7-da632d0c701e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:56.747116 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 146/283] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:56.771902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea33eba-52a1-4950-8c05-3c28b72620b4 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:57.052946 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea33eba-52a1-4950-8c05-3c28b72620b4 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:57.101588 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7a5b7-55ac-457c-95c7-da632d0c701e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:57.125853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:57.136275 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f89a23-c5cf-4f7d-a008-a89636896731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:57.137414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f89a23-c5cf-4f7d-a008-a89636896731 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:57.137916 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f89a23-c5cf-4f7d-a008-a89636896731 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:57.138411 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f89a23-c5cf-4f7d-a008-a89636896731 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:57.181642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f89a23-c5cf-4f7d-a008-a89636896731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:57.198285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:57.201521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-81f89a23-c5cf-4f7d-a008-a89636896731 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:08:57.201521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:57.201521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:57.201521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:57.201521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:57.201521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:57.201521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:57.201521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:57.201521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:57.201521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:57.201521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:57.201521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:57.201521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:57.201521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:57.201521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:57.201521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:57.202427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:57.202427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:57.202427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:57.202427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:57.202427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:57.202427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:57.202427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:57.202427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:57.202427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:57.202427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:57.202427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:57.202427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:57.202427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:57.202427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:57.202427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:57.202427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:57.203239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:57.203239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:57.203239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:57.203239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:57.203239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:57.203239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:57.203239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:57.203239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:57.203239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:57.203239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:57.203239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:08:57.203239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:08:57.203838 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 138/284] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:57.224691 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f89a23-c5cf-4f7d-a008-a89636896731 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:57.431070 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7a5b7-55ac-457c-95c7-da632d0c701e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:57.531361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f89a23-c5cf-4f7d-a008-a89636896731 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:57.535831 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf5cb2-507f-44b4-b69b-6765dabec13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:57.536971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf5cb2-507f-44b4-b69b-6765dabec13e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:57.537506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf5cb2-507f-44b4-b69b-6765dabec13e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:57.538001 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf5cb2-507f-44b4-b69b-6765dabec13e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:57.584156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf5cb2-507f-44b4-b69b-6765dabec13e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:57.602054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-0ddf5cb2-507f-44b4-b69b-6765dabec13e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:57.602054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:57.602054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:57.602054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:57.602054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:57.602054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:57.602054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:57.602054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:57.602054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:57.602054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:57.602054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:57.602054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:57.602054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:57.602054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:57.602054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:57.602054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:57.603006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:57.603006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:57.603006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:57.603006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:57.603006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:57.603006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:57.603006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:57.603006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:57.603006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:57.603006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:57.603006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:57.603006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:57.603006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:57.603006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:57.603006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:57.603006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:57.603789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:57.603789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:57.603789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:57.603789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:57.603789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:57.603789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:57.603789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:57.603789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:57.603789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:57.603789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:57.603789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:08:57.603789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:57.604390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 147/285] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 505 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:57.614369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf5cb2-507f-44b4-b69b-6765dabec13e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:57.658760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f25d49-c019-45b2-b79d-f0c027582922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:57.659875 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f25d49-c019-45b2-b79d-f0c027582922 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:57.660296 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f25d49-c019-45b2-b79d-f0c027582922 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:57.660715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f25d49-c019-45b2-b79d-f0c027582922 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:57.661809 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f25d49-c019-45b2-b79d-f0c027582922 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:57.662280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f25d49-c019-45b2-b79d-f0c027582922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:57.664605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f25d49-c019-45b2-b79d-f0c027582922 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:57.694902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 139/286] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:08:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 471 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:57.701852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f25d49-c019-45b2-b79d-f0c027582922 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:57.839532 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:57.850636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf5cb2-507f-44b4-b69b-6765dabec13e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:57.868106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:57.932724 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e7790-3c7f-493b-bae0-6d390118e6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:57.933774 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e7790-3c7f-493b-bae0-6d390118e6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:57.942013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e7790-3c7f-493b-bae0-6d390118e6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:57.942586 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e7790-3c7f-493b-bae0-6d390118e6e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:57.943843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e7790-3c7f-493b-bae0-6d390118e6e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:57.944346 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e7790-3c7f-493b-bae0-6d390118e6e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:57.949427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e7790-3c7f-493b-bae0-6d390118e6e2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:57.986296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 148/287] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:08:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:57.993481 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e7790-3c7f-493b-bae0-6d390118e6e2 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:58.091289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f25d49-c019-45b2-b79d-f0c027582922 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:58.179213 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b9b2e-879d-4408-a264-236812927c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:58.179902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b9b2e-879d-4408-a264-236812927c44 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:58.182396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b9b2e-879d-4408-a264-236812927c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:58.182396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b9b2e-879d-4408-a264-236812927c44 None admin] PATH_INFO: `/v3/users/5a0c2c9466964d0d8ca3686550dd5709` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:58.183320 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b9b2e-879d-4408-a264-236812927c44 None admin] RBAC: Authorizing `identity:get_user(user_id=5a0c2c9466964d0d8ca3686550dd5709)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:58.184872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b9b2e-879d-4408-a264-236812927c44 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:58.186686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 140/288] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:57 2022] GET /identity/v3/users/5a0c2c9466964d0d8ca3686550dd5709 => generated 297 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:58.196824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b9b2e-879d-4408-a264-236812927c44 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:58.285289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e7790-3c7f-493b-bae0-6d390118e6e2 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:58.382918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418393a9-ca48-48b3-972b-b8705051c166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:58.383933 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418393a9-ca48-48b3-972b-b8705051c166 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:58.384396 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418393a9-ca48-48b3-972b-b8705051c166 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:58.384876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418393a9-ca48-48b3-972b-b8705051c166 None admin] PATH_INFO: `/v3/users/ebf4437ca15744af87860f17b78d8c96` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:58.387071 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418393a9-ca48-48b3-972b-b8705051c166 None admin] RBAC: Authorizing `identity:get_user(user_id=ebf4437ca15744af87860f17b78d8c96)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:58.389791 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418393a9-ca48-48b3-972b-b8705051c166 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:58.392959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 149/289] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:57 2022] GET /identity/v3/users/ebf4437ca15744af87860f17b78d8c96 => generated 297 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:58.395784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976778d-cc64-4c36-999a-e202089743c5 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:58.396214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976778d-cc64-4c36-999a-e202089743c5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:58.396713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976778d-cc64-4c36-999a-e202089743c5 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:58.407408 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 150/290] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:58 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:58.408890 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976778d-cc64-4c36-999a-e202089743c5 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:58.430808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b9b2e-879d-4408-a264-236812927c44 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:58.439597 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:58.455731 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:58.484895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a31ab-a4d1-4523-ba05-15a342dd759e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:58.485629 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a31ab-a4d1-4523-ba05-15a342dd759e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:58.485986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a31ab-a4d1-4523-ba05-15a342dd759e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:58.486330 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a31ab-a4d1-4523-ba05-15a342dd759e None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:58.487879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a31ab-a4d1-4523-ba05-15a342dd759e None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:58.490379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a31ab-a4d1-4523-ba05-15a342dd759e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:58.492413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 141/291] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:58 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:58.497966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92800c8-8816-43cc-a911-dd662a34828a None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:58.498345 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92800c8-8816-43cc-a911-dd662a34828a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:58.498613 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92800c8-8816-43cc-a911-dd662a34828a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:58.523770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-b92800c8-8816-43cc-a911-dd662a34828a None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:58.525858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-b92800c8-8816-43cc-a911-dd662a34828a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:58.601993 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976778d-cc64-4c36-999a-e202089743c5 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:58.622253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 142/292] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:58.623568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92800c8-8816-43cc-a911-dd662a34828a None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:58.684837 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b2f59-87c1-4c7e-8d97-025e7e41b9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:58.686033 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b2f59-87c1-4c7e-8d97-025e7e41b9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:58.686597 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b2f59-87c1-4c7e-8d97-025e7e41b9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:58.687112 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b2f59-87c1-4c7e-8d97-025e7e41b9a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:58.837611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92800c8-8816-43cc-a911-dd662a34828a None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:58.861408 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b2f59-87c1-4c7e-8d97-025e7e41b9a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:58.864900 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:58.865142 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b2f59-87c1-4c7e-8d97-025e7e41b9a9 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:58.903921 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22056c-7605-4367-9860-36a5656512ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:58.905435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22056c-7605-4367-9860-36a5656512ee None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:58.905912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22056c-7605-4367-9860-36a5656512ee None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:58.906371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22056c-7605-4367-9860-36a5656512ee None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a/users/5a0c2c9466964d0d8ca3686550dd5709/roles/8d5b36826f914e958bd80fc21156e286` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:58.907977 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:58.913811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22056c-7605-4367-9860-36a5656512ee None admin] RBAC: Authorizing `identity:create_grant(project_id=dc2c956c8a3f4d0f832f927cdc74095a, user_id=5a0c2c9466964d0d8ca3686550dd5709, role_id=8d5b36826f914e958bd80fc21156e286)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:58.915520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22056c-7605-4367-9860-36a5656512ee None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:58.952396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 143/293] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:08:58 2022] PUT /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a/users/5a0c2c9466964d0d8ca3686550dd5709/roles/8d5b36826f914e958bd80fc21156e286 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:08:58.954489 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74cdae-76b7-4acb-ae9b-0221deac4474 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:58.954929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74cdae-76b7-4acb-ae9b-0221deac4474 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:58.955395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74cdae-76b7-4acb-ae9b-0221deac4474 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:58.958467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 144/294] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:58.960169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9aa931-08ae-4c7a-91bc-2a10a954bf95 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:58.960593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9aa931-08ae-4c7a-91bc-2a10a954bf95 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:58.961007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9aa931-08ae-4c7a-91bc-2a10a954bf95 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:58.969109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 145/295] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:08:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:58.969109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171066a-203d-46f3-9d5d-f308aae48e99 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:58.969109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171066a-203d-46f3-9d5d-f308aae48e99 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:58.969109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171066a-203d-46f3-9d5d-f308aae48e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:59.006126 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-9171066a-203d-46f3-9d5d-f308aae48e99 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:59.008509 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-9171066a-203d-46f3-9d5d-f308aae48e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:59.067765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 151/296] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:08:59.067765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b4ced-874b-487b-adfa-8c9e5aef2b90 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:59.067765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b4ced-874b-487b-adfa-8c9e5aef2b90 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:59.067765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b4ced-874b-487b-adfa-8c9e5aef2b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:59.087551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-642b4ced-874b-487b-adfa-8c9e5aef2b90 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:59.089841 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-642b4ced-874b-487b-adfa-8c9e5aef2b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:59.164988 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 146/297] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:59.167100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171066a-203d-46f3-9d5d-f308aae48e99 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:59.208481 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 152/298] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:59.212789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b4ced-874b-487b-adfa-8c9e5aef2b90 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:59.308271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:59.325117 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171066a-203d-46f3-9d5d-f308aae48e99 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:59.328021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:59.401086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0566f-9a40-4953-ad8d-82cfef6636e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:59.401086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0566f-9a40-4953-ad8d-82cfef6636e3 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:59.401086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0566f-9a40-4953-ad8d-82cfef6636e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:59.401086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0566f-9a40-4953-ad8d-82cfef6636e3 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:59.405088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0566f-9a40-4953-ad8d-82cfef6636e3 None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:59.413088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0566f-9a40-4953-ad8d-82cfef6636e3 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:59.418221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 147/299] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:08:59 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:59.427817 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0566f-9a40-4953-ad8d-82cfef6636e3 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:59.497599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b4ced-874b-487b-adfa-8c9e5aef2b90 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:59.609695 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00810a2c-b46e-412d-be2a-d41ed093c8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:59.610369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00810a2c-b46e-412d-be2a-d41ed093c8ac None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:59.610644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00810a2c-b46e-412d-be2a-d41ed093c8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:59.610954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00810a2c-b46e-412d-be2a-d41ed093c8ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:59.642920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00810a2c-b46e-412d-be2a-d41ed093c8ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:59.652642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-00810a2c-b46e-412d-be2a-d41ed093c8ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:08:59.652642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:59.652642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:59.652642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:59.652642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:59.652642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:59.652642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:59.652642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:59.652642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:59.652642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:59.652642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:59.652642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:59.652642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:59.652642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:59.652642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:59.652642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:59.654872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:59.654872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:59.654872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:59.654872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:59.654872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:59.654872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:59.654872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:59.654872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:59.654872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:59.654872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:59.654872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:59.654872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:59.654872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:59.654872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:59.654872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:59.654872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:59.656488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:59.656488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:59.656488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:59.656488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:59.656488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:59.656488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:59.656488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:59.656488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:59.656488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:59.656488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:59.656488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:08:59.656488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:08:59.656488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 153/300] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 445 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:59.669227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00810a2c-b46e-412d-be2a-d41ed093c8ac None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:59.692171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0566f-9a40-4953-ad8d-82cfef6636e3 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:59.744301 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99226903-b36d-4074-9402-05ff9742a816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:59.744962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99226903-b36d-4074-9402-05ff9742a816 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:59.745304 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99226903-b36d-4074-9402-05ff9742a816 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:59.745539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99226903-b36d-4074-9402-05ff9742a816 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:59.746524 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99226903-b36d-4074-9402-05ff9742a816 None admin] RBAC: Adding query filter params (role.id=fad7ba55962b40459b40dcc2bad395b2, scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a, user.id=ebf4437ca15744af87860f17b78d8c96) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:59.747133 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99226903-b36d-4074-9402-05ff9742a816 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:59.748543 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99226903-b36d-4074-9402-05ff9742a816 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:59.756841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 148/301] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:08:59 2022] GET /identity/v3/role_assignments?user.id=ebf4437ca15744af87860f17b78d8c96&scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a&role.id=fad7ba55962b40459b40dcc2bad395b2 => generated 608 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:59.757776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99226903-b36d-4074-9402-05ff9742a816 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:59.849762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:59.918704 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00810a2c-b46e-412d-be2a-d41ed093c8ac None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:59.934485 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:59.989096 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99226903-b36d-4074-9402-05ff9742a816 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:00.006053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2e851-5c86-4529-8a0a-f8ee2da2c6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:00.007059 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2e851-5c86-4529-8a0a-f8ee2da2c6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:00.007511 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2e851-5c86-4529-8a0a-f8ee2da2c6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:00.007963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2e851-5c86-4529-8a0a-f8ee2da2c6f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:00.009179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2e851-5c86-4529-8a0a-f8ee2da2c6f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:00.009664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2e851-5c86-4529-8a0a-f8ee2da2c6f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:00.012183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2e851-5c86-4529-8a0a-f8ee2da2c6f4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:00.029240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 154/302] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:08:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:00.031829 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2e851-5c86-4529-8a0a-f8ee2da2c6f4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:00.066576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65922fb1-ae18-499a-9f59-ba49809a7685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:00.070093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65922fb1-ae18-499a-9f59-ba49809a7685 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:00.070851 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65922fb1-ae18-499a-9f59-ba49809a7685 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:00.071747 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65922fb1-ae18-499a-9f59-ba49809a7685 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:00.074297 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65922fb1-ae18-499a-9f59-ba49809a7685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:00.075827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65922fb1-ae18-499a-9f59-ba49809a7685 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:00.077712 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 149/303] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:00.078953 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65922fb1-ae18-499a-9f59-ba49809a7685 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:00.216478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2e851-5c86-4529-8a0a-f8ee2da2c6f4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:00.266958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65922fb1-ae18-499a-9f59-ba49809a7685 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:00.270461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d826e96-6e72-44cc-ab53-63fb8eadc68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:00.271390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d826e96-6e72-44cc-ab53-63fb8eadc68d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:00.271499 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:00.271577 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d826e96-6e72-44cc-ab53-63fb8eadc68d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:00.271646 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d826e96-6e72-44cc-ab53-63fb8eadc68d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:00.288529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d826e96-6e72-44cc-ab53-63fb8eadc68d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:00.296036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-9d826e96-6e72-44cc-ab53-63fb8eadc68d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:00.296036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:00.296036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:00.296036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:00.296036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:00.296036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:00.296036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:00.296036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:00.296036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:00.296036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:00.296036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:00.296036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:00.296036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:00.296036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:00.296036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:00.296036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:00.298157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:00.298157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:00.298157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:00.298157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:00.298157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:00.298157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:00.298157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:00.298157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:00.298157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:00.298157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:00.298157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:00.298157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:00.298157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:00.298157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:00.298157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:00.298157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:00.299729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:00.299729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:00.299729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:00.299729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:00.299729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:00.299729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:00.299729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:00.299729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:00.299729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:00.299729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:00.299729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:00.299729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:00.299729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 155/304] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:00.299729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d826e96-6e72-44cc-ab53-63fb8eadc68d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:00.335783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516fd95-2189-480d-8075-ddd2fb0f0633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:00.336935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516fd95-2189-480d-8075-ddd2fb0f0633 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:00.337546 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516fd95-2189-480d-8075-ddd2fb0f0633 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:00.337818 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516fd95-2189-480d-8075-ddd2fb0f0633 None admin] PATH_INFO: `/v3/users/5d722b0c96414362a7c81067e6c6d170` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:00.351299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:00.362064 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516fd95-2189-480d-8075-ddd2fb0f0633 None admin] RBAC: Authorizing `identity:get_user(user_id=5d722b0c96414362a7c81067e6c6d170)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:00.364593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516fd95-2189-480d-8075-ddd2fb0f0633 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:00.374515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 150/305] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:00 2022] GET /identity/v3/users/5d722b0c96414362a7c81067e6c6d170 => generated 293 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:00.374515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516fd95-2189-480d-8075-ddd2fb0f0633 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:00.500182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d826e96-6e72-44cc-ab53-63fb8eadc68d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:00.545756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516fd95-2189-480d-8075-ddd2fb0f0633 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:00.576778 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161d094-6931-47c0-a4e8-852806f38517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:00.577793 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161d094-6931-47c0-a4e8-852806f38517 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:00.578218 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161d094-6931-47c0-a4e8-852806f38517 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:00.578639 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161d094-6931-47c0-a4e8-852806f38517 None admin] PATH_INFO: `/v3/groups` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:00.579668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161d094-6931-47c0-a4e8-852806f38517 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:00.583087 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161d094-6931-47c0-a4e8-852806f38517 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:00.601974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6242e-2377-4454-926f-e09bdbaf12e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:00.602753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6242e-2377-4454-926f-e09bdbaf12e6 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:00.603120 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6242e-2377-4454-926f-e09bdbaf12e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:00.603522 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6242e-2377-4454-926f-e09bdbaf12e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:00.604305 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6242e-2377-4454-926f-e09bdbaf12e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:00.604733 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6242e-2377-4454-926f-e09bdbaf12e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:00.607099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6242e-2377-4454-926f-e09bdbaf12e6 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:00.625226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 156/306] 10.222.0.88 () {60 vars in 1134 bytes} [Tue Aug 23 22:09:00 2022] POST /identity/v3/groups => generated 229 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:09:00.625226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161d094-6931-47c0-a4e8-852806f38517 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:00.630207 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 151/307] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:00.641088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6242e-2377-4454-926f-e09bdbaf12e6 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:00.684467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:00.836020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:00.840694 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6242e-2377-4454-926f-e09bdbaf12e6 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:00.864385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161d094-6931-47c0-a4e8-852806f38517 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:00.903087 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28248faa-49e6-426f-a2ec-8d5431cf3b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:00.904673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28248faa-49e6-426f-a2ec-8d5431cf3b31 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:00.905164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28248faa-49e6-426f-a2ec-8d5431cf3b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:00.905451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28248faa-49e6-426f-a2ec-8d5431cf3b31 None admin] PATH_INFO: `/v3/users/4ed7d09ca57f4bb6a2f19e748b430dd4` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:00.907604 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28248faa-49e6-426f-a2ec-8d5431cf3b31 None admin] RBAC: Authorizing `identity:get_user(user_id=4ed7d09ca57f4bb6a2f19e748b430dd4)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:00.910154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28248faa-49e6-426f-a2ec-8d5431cf3b31 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:00.917118 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 152/308] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:00 2022] GET /identity/v3/users/4ed7d09ca57f4bb6a2f19e748b430dd4 => generated 293 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:00.919298 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28248faa-49e6-426f-a2ec-8d5431cf3b31 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:00.921656 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925cd59-8a89-427b-8aab-945fb40729ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:00.922288 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925cd59-8a89-427b-8aab-945fb40729ee None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:00.922737 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925cd59-8a89-427b-8aab-945fb40729ee None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:00.923121 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925cd59-8a89-427b-8aab-945fb40729ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:00.923873 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925cd59-8a89-427b-8aab-945fb40729ee None admin] RBAC: Adding query filter params (role.id=8d5b36826f914e958bd80fc21156e286, user.id=5d722b0c96414362a7c81067e6c6d170, scope.system=all) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:00.924183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925cd59-8a89-427b-8aab-945fb40729ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:00.928483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925cd59-8a89-427b-8aab-945fb40729ee None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:00.936055 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 157/309] 10.222.0.88 () {56 vars in 1308 bytes} [Tue Aug 23 22:09:00 2022] GET /identity/v3/role_assignments?user.id=5d722b0c96414362a7c81067e6c6d170&scope.system=all&role.id=8d5b36826f914e958bd80fc21156e286 => generated 226 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:01.063504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:01.077702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28248faa-49e6-426f-a2ec-8d5431cf3b31 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:01.127537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe28c5-aa3e-4afb-9ad9-3cd148630570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:01.128804 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe28c5-aa3e-4afb-9ad9-3cd148630570 None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:01.129497 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe28c5-aa3e-4afb-9ad9-3cd148630570 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:01.129986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe28c5-aa3e-4afb-9ad9-3cd148630570 None admin] PATH_INFO: `/v3/system/users/4ed7d09ca57f4bb6a2f19e748b430dd4/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:01.131958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe28c5-aa3e-4afb-9ad9-3cd148630570 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4ed7d09ca57f4bb6a2f19e748b430dd4, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:01.133703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe28c5-aa3e-4afb-9ad9-3cd148630570 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:01.147499 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 153/310] 10.222.0.88 () {58 vars in 1334 bytes} [Tue Aug 23 22:09:00 2022] PUT /identity/v3/system/users/4ed7d09ca57f4bb6a2f19e748b430dd4/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:01.908407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdad72-b439-4efc-a3f8-deb20db7bd6b None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:01.908823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdad72-b439-4efc-a3f8-deb20db7bd6b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:01.908971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdad72-b439-4efc-a3f8-deb20db7bd6b None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:01.911369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 158/311] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:09:01.918196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea3074-2531-4ff8-a87e-02dcb525d5f5 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:01.918981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea3074-2531-4ff8-a87e-02dcb525d5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:01.919281 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea3074-2531-4ff8-a87e-02dcb525d5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:01.949163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-41ea3074-2531-4ff8-a87e-02dcb525d5f5 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:01.951581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-41ea3074-2531-4ff8-a87e-02dcb525d5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:02.088873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 154/312] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:02.504401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdad72-b439-4efc-a3f8-deb20db7bd6b None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:02.650883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdad72-b439-4efc-a3f8-deb20db7bd6b None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:02.695959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5083b-9455-445d-adf3-e273caa7ca04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:02.696614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5083b-9455-445d-adf3-e273caa7ca04 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:02.696957 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5083b-9455-445d-adf3-e273caa7ca04 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:02.697277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5083b-9455-445d-adf3-e273caa7ca04 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:02.711330 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5083b-9455-445d-adf3-e273caa7ca04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:02.716318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-36d5083b-9455-445d-adf3-e273caa7ca04 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:02.716318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:02.716318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:02.716318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:02.716318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:02.716318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:02.716318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:02.716318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:02.716318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:02.716318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:02.716318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:02.716318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:02.716318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:02.716318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:02.716318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:02.716318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:02.717334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:02.717334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:02.717334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:02.717334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:02.717334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:02.717334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:02.717334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:02.717334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:02.717334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:02.717334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:02.717334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:02.717334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:02.717334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:02.717334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:02.717334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:02.717334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:02.718130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:02.718130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:02.718130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:02.718130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:02.718130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:02.718130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:02.718130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:02.718130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:02.718130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:02.718130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:02.718130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:02.718130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:02.718130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 159/313] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:02.726243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea3074-2531-4ff8-a87e-02dcb525d5f5 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:02.857811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:02.890902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea3074-2531-4ff8-a87e-02dcb525d5f5 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:02.963629 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1b86b-5cdf-4c03-b526-1ef4645d18f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:02.964683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1b86b-5cdf-4c03-b526-1ef4645d18f4 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:02.965172 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1b86b-5cdf-4c03-b526-1ef4645d18f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:02.965624 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1b86b-5cdf-4c03-b526-1ef4645d18f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:02.966844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1b86b-5cdf-4c03-b526-1ef4645d18f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:02.967415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1b86b-5cdf-4c03-b526-1ef4645d18f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:02.970014 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1b86b-5cdf-4c03-b526-1ef4645d18f4 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:02.990860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 155/314] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:02.998734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5083b-9455-445d-adf3-e273caa7ca04 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:03.020998 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:03.154466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5083b-9455-445d-adf3-e273caa7ca04 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:03.205787 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d67d9-2525-4880-9ab7-bb1ddea54dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:03.206951 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d67d9-2525-4880-9ab7-bb1ddea54dae None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:03.207460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d67d9-2525-4880-9ab7-bb1ddea54dae None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:03.207940 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d67d9-2525-4880-9ab7-bb1ddea54dae None admin] PATH_INFO: `/v3/users/5a0c2c9466964d0d8ca3686550dd5709` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:03.209983 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d67d9-2525-4880-9ab7-bb1ddea54dae None admin] RBAC: Authorizing `identity:get_user(user_id=5a0c2c9466964d0d8ca3686550dd5709)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:03.212633 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d67d9-2525-4880-9ab7-bb1ddea54dae None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:03.216310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 160/315] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:02 2022] GET /identity/v3/users/5a0c2c9466964d0d8ca3686550dd5709 => generated 297 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:03.223076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1b86b-5cdf-4c03-b526-1ef4645d18f4 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:03.387359 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1b86b-5cdf-4c03-b526-1ef4645d18f4 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:03.434614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90096573-153e-49a8-bc46-ecd7e980975b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:03.435161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90096573-153e-49a8-bc46-ecd7e980975b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:03.435448 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90096573-153e-49a8-bc46-ecd7e980975b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:03.435725 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90096573-153e-49a8-bc46-ecd7e980975b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:03.468733 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:03.533806 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f2d43-793b-4ce9-8e4a-2a838996b5c6 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:03.535094 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f2d43-793b-4ce9-8e4a-2a838996b5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:03.535276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f2d43-793b-4ce9-8e4a-2a838996b5c6 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:03.537964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 161/316] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:03.547430 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d468a1-ead6-4023-a78d-d56ad13b41f2 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:03.547963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d468a1-ead6-4023-a78d-d56ad13b41f2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:03.548461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d468a1-ead6-4023-a78d-d56ad13b41f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:03.583985 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-51d468a1-ead6-4023-a78d-d56ad13b41f2 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:03.587253 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-51d468a1-ead6-4023-a78d-d56ad13b41f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:03.591341 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90096573-153e-49a8-bc46-ecd7e980975b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:03.595103 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90096573-153e-49a8-bc46-ecd7e980975b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:03.745735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 162/317] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:03.784963 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 156/318] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:03.798602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d468a1-ead6-4023-a78d-d56ad13b41f2 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:03.825700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:03.833152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc900f6-46ad-49ab-8393-34f8d316d730 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:03.833464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc900f6-46ad-49ab-8393-34f8d316d730 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:03.833698 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc900f6-46ad-49ab-8393-34f8d316d730 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:03.837218 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 157/319] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:09:03.843690 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbb8d7-2a58-4aa3-8b1e-b13a22556c5b None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:03.844070 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbb8d7-2a58-4aa3-8b1e-b13a22556c5b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:03.844349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbb8d7-2a58-4aa3-8b1e-b13a22556c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:03.866440 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-58bbb8d7-2a58-4aa3-8b1e-b13a22556c5b None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:03.867720 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-58bbb8d7-2a58-4aa3-8b1e-b13a22556c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:03.943429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:03.974207 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d468a1-ead6-4023-a78d-d56ad13b41f2 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:03.981185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 158/320] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:03.990997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb0127-7e27-466f-a202-84f95896988e None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:03.991974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb0127-7e27-466f-a202-84f95896988e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:03.992744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb0127-7e27-466f-a202-84f95896988e None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:03.996345 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 159/321] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:03 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:04.005104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9855972-1843-4c49-93fe-b3fdb131e350 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:04.005550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9855972-1843-4c49-93fe-b3fdb131e350 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:04.005965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9855972-1843-4c49-93fe-b3fdb131e350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:04.036195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-a9855972-1843-4c49-93fe-b3fdb131e350 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:04.038785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a9855972-1843-4c49-93fe-b3fdb131e350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:04.050378 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757112e7-63a4-46bb-9700-99fdd2ffc6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:04.051549 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757112e7-63a4-46bb-9700-99fdd2ffc6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:04.052023 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757112e7-63a4-46bb-9700-99fdd2ffc6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:04.052494 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757112e7-63a4-46bb-9700-99fdd2ffc6c7 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:04.054906 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757112e7-63a4-46bb-9700-99fdd2ffc6c7 None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:04.058345 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757112e7-63a4-46bb-9700-99fdd2ffc6c7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:04.065563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 163/322] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:03 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:04.070213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757112e7-63a4-46bb-9700-99fdd2ffc6c7 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:04.181271 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 160/323] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:04.182481 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9855972-1843-4c49-93fe-b3fdb131e350 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:04.249335 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:04.295032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757112e7-63a4-46bb-9700-99fdd2ffc6c7 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:04.363917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bec78-6a86-4945-b9bc-13aa7d01ba3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:04.365479 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bec78-6a86-4945-b9bc-13aa7d01ba3d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:04.366226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bec78-6a86-4945-b9bc-13aa7d01ba3d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:04.366886 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9855972-1843-4c49-93fe-b3fdb131e350 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:04.367299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bec78-6a86-4945-b9bc-13aa7d01ba3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:04.368899 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9bec78-6a86-4945-b9bc-13aa7d01ba3d None admin] RBAC: Adding query filter params (role.id=8d5b36826f914e958bd80fc21156e286, scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a, user.id=5a0c2c9466964d0d8ca3686550dd5709) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:04.369851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9bec78-6a86-4945-b9bc-13aa7d01ba3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:04.372883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9bec78-6a86-4945-b9bc-13aa7d01ba3d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:04.384761 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:04.391228 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 164/324] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:04 2022] GET /identity/v3/role_assignments?user.id=5a0c2c9466964d0d8ca3686550dd5709&scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a&role.id=8d5b36826f914e958bd80fc21156e286 => generated 608 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:04.414391 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bec78-6a86-4945-b9bc-13aa7d01ba3d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:04.434892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76511fe-e4cb-4680-b454-c2ac6048c3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:04.435907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76511fe-e4cb-4680-b454-c2ac6048c3cf None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:04.436364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76511fe-e4cb-4680-b454-c2ac6048c3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:04.436809 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76511fe-e4cb-4680-b454-c2ac6048c3cf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:04.457195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76511fe-e4cb-4680-b454-c2ac6048c3cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:04.465145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-d76511fe-e4cb-4680-b454-c2ac6048c3cf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:04.465145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:04.465145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:04.465145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:04.465145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:04.465145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:04.465145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:04.465145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:04.465145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:04.465145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:04.465145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:04.465145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:04.465145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:04.465145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:04.465145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:04.465145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:04.466892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:04.466892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:04.466892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:04.466892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:04.466892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:04.466892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:04.466892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:04.466892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:04.466892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:04.466892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:04.466892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:04.466892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:04.466892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:04.466892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:04.466892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:04.466892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:04.469288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:04.469288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:04.469288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:04.469288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:04.469288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:04.469288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:04.469288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:04.469288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:04.469288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:04.469288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:04.469288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:04.469288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:04.469288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 161/325] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:04.476700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76511fe-e4cb-4680-b454-c2ac6048c3cf None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:04.641754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bec78-6a86-4945-b9bc-13aa7d01ba3d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:04.664962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76511fe-e4cb-4680-b454-c2ac6048c3cf None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:04.712511 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ccf21-9593-46e2-95b6-c754391213c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:04.714230 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ccf21-9593-46e2-95b6-c754391213c6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:04.714230 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ccf21-9593-46e2-95b6-c754391213c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:04.714553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ccf21-9593-46e2-95b6-c754391213c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:04.724650 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ad5bb-69ae-4ada-907b-fef0767885d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:04.725647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ad5bb-69ae-4ada-907b-fef0767885d4 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:04.726094 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ad5bb-69ae-4ada-907b-fef0767885d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:04.726537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ad5bb-69ae-4ada-907b-fef0767885d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:04.727638 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ad5bb-69ae-4ada-907b-fef0767885d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:04.728150 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ad5bb-69ae-4ada-907b-fef0767885d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:04.730438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ad5bb-69ae-4ada-907b-fef0767885d4 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:04.735367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ccf21-9593-46e2-95b6-c754391213c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:04.743785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-216ccf21-9593-46e2-95b6-c754391213c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:04.743785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:04.743785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:04.743785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:04.743785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:04.743785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:04.743785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:04.743785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:04.743785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:04.743785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:04.743785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:04.743785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:04.743785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:04.743785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:04.743785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:04.743785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:04.752274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:04.752274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:04.752274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:04.752274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:04.752274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:04.752274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:04.752274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:04.752274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:04.752274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:04.752274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:04.752274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:04.752274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:04.752274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:04.752274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:04.752274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:04.752274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:04.754434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:04.754434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:04.754434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:04.754434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:04.754434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:04.754434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:04.754434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:04.754434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:04.754434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:04.754434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:04.754434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:04.754434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:04.755589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 162/327] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:04.755718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 165/327] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:09:04.755878 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ad5bb-69ae-4ada-907b-fef0767885d4 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:04.756005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ccf21-9593-46e2-95b6-c754391213c6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:04.758500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:04.795775 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:04.903344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ccf21-9593-46e2-95b6-c754391213c6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:04.911674 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ad5bb-69ae-4ada-907b-fef0767885d4 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:04.964091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2acb48-2c7b-4363-b598-0bf832263613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:04.964761 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2acb48-2c7b-4363-b598-0bf832263613 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:04.965122 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2acb48-2c7b-4363-b598-0bf832263613 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:04.965406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2acb48-2c7b-4363-b598-0bf832263613 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:04.965926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8282b-d714-458d-87ec-0749f5abe863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:04.967023 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8282b-d714-458d-87ec-0749f5abe863 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:04.967152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2acb48-2c7b-4363-b598-0bf832263613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:04.967507 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8282b-d714-458d-87ec-0749f5abe863 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:04.967966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8282b-d714-458d-87ec-0749f5abe863 None admin] PATH_INFO: `/v3/users/5d722b0c96414362a7c81067e6c6d170` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:04.968716 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2acb48-2c7b-4363-b598-0bf832263613 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:04.970134 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8282b-d714-458d-87ec-0749f5abe863 None admin] RBAC: Authorizing `identity:get_user(user_id=5d722b0c96414362a7c81067e6c6d170)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:04.970563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 163/328] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:04.971525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2acb48-2c7b-4363-b598-0bf832263613 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:04.974494 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8282b-d714-458d-87ec-0749f5abe863 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:04.977885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 166/329] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:04 2022] GET /identity/v3/users/5d722b0c96414362a7c81067e6c6d170 => generated 293 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:04.986388 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8282b-d714-458d-87ec-0749f5abe863 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:05.092669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:05.127644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:05.133525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2acb48-2c7b-4363-b598-0bf832263613 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:05.141699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8282b-d714-458d-87ec-0749f5abe863 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:05.186169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d8448-376d-4b95-9ad5-5d5789855792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:05.186702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d8448-376d-4b95-9ad5-5d5789855792 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:05.186981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d8448-376d-4b95-9ad5-5d5789855792 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:05.187249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d8448-376d-4b95-9ad5-5d5789855792 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:05.187992 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d8448-376d-4b95-9ad5-5d5789855792 None admin] RBAC: Adding query filter params (name=member) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:05.188222 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d8448-376d-4b95-9ad5-5d5789855792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:05.189657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d8448-376d-4b95-9ad5-5d5789855792 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:05.196229 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329e750-595c-48be-b753-511f5e106dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:05.197218 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329e750-595c-48be-b753-511f5e106dc3 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:05.197585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329e750-595c-48be-b753-511f5e106dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:05.198049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329e750-595c-48be-b753-511f5e106dc3 None admin] PATH_INFO: `/v3/groups` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:05.198816 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c329e750-595c-48be-b753-511f5e106dc3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:05.200402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c329e750-595c-48be-b753-511f5e106dc3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:05.205075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 164/330] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:05.207187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d8448-376d-4b95-9ad5-5d5789855792 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:05.230483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 167/331] 10.222.0.88 () {60 vars in 1134 bytes} [Tue Aug 23 22:09:04 2022] POST /identity/v3/groups => generated 226 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:09:05.231098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329e750-595c-48be-b753-511f5e106dc3 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:05.394318 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d8448-376d-4b95-9ad5-5d5789855792 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:05.411900 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329e750-595c-48be-b753-511f5e106dc3 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:05.447433 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88a016-9b13-4075-b5e6-8ae81ef125d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:05.447983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88a016-9b13-4075-b5e6-8ae81ef125d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:05.448256 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88a016-9b13-4075-b5e6-8ae81ef125d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:05.449225 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88a016-9b13-4075-b5e6-8ae81ef125d2 None admin] PATH_INFO: `/v3/system/users/5d722b0c96414362a7c81067e6c6d170/roles/8d5b36826f914e958bd80fc21156e286` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:05.450835 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88a016-9b13-4075-b5e6-8ae81ef125d2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5d722b0c96414362a7c81067e6c6d170, role_id=8d5b36826f914e958bd80fc21156e286)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:05.452255 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88a016-9b13-4075-b5e6-8ae81ef125d2 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:05.465696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:05.468346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 165/332] 10.222.0.88 () {58 vars in 1334 bytes} [Tue Aug 23 22:09:05 2022] PUT /identity/v3/system/users/5d722b0c96414362a7c81067e6c6d170/roles/8d5b36826f914e958bd80fc21156e286 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:05.477944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df38c4c-7556-4a1c-a553-7f1eeb07801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:05.479167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df38c4c-7556-4a1c-a553-7f1eeb07801a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:05.479894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df38c4c-7556-4a1c-a553-7f1eeb07801a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:05.480256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df38c4c-7556-4a1c-a553-7f1eeb07801a None admin] PATH_INFO: `/v3/users/4ed7d09ca57f4bb6a2f19e748b430dd4` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:05.482352 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df38c4c-7556-4a1c-a553-7f1eeb07801a None admin] RBAC: Authorizing `identity:get_user(user_id=4ed7d09ca57f4bb6a2f19e748b430dd4)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:05.485204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df38c4c-7556-4a1c-a553-7f1eeb07801a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:05.493461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 168/333] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:05 2022] GET /identity/v3/users/4ed7d09ca57f4bb6a2f19e748b430dd4 => generated 293 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:05.496821 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88a016-9b13-4075-b5e6-8ae81ef125d2 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:05.548818 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:05.711581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88a016-9b13-4075-b5e6-8ae81ef125d2 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:05.778282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16381b2-a18f-4ee7-ac0e-0621c5af42d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:05.779919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16381b2-a18f-4ee7-ac0e-0621c5af42d4 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:05.780731 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16381b2-a18f-4ee7-ac0e-0621c5af42d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:05.781610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16381b2-a18f-4ee7-ac0e-0621c5af42d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:05.783260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16381b2-a18f-4ee7-ac0e-0621c5af42d4 None admin] RBAC: Adding query filter params (role.id=fad7ba55962b40459b40dcc2bad395b2, user.id=4ed7d09ca57f4bb6a2f19e748b430dd4, scope.system=all) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:05.784169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16381b2-a18f-4ee7-ac0e-0621c5af42d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:05.787348 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16381b2-a18f-4ee7-ac0e-0621c5af42d4 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:05.813100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 166/334] 10.222.0.88 () {56 vars in 1308 bytes} [Tue Aug 23 22:09:05 2022] GET /identity/v3/role_assignments?user.id=4ed7d09ca57f4bb6a2f19e748b430dd4&scope.system=all&role.id=fad7ba55962b40459b40dcc2bad395b2 => generated 510 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:08.953026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf054b2-24c1-4b85-b611-09c6de5a3733 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:08.954574 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf054b2-24c1-4b85-b611-09c6de5a3733 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:08.955316 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf054b2-24c1-4b85-b611-09c6de5a3733 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:08.958903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 169/335] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:08.975934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ee54-f630-49d9-9a4f-cc4efb88d158 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:08.977384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ee54-f630-49d9-9a4f-cc4efb88d158 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:08.978128 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ee54-f630-49d9-9a4f-cc4efb88d158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:09.017270 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-74f7ee54-f630-49d9-9a4f-cc4efb88d158 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:09.019868 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-74f7ee54-f630-49d9-9a4f-cc4efb88d158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:09.177769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 167/336] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:09.229455 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2f6c1-d9d1-4cc3-991b-8fb97f4696ee None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:09.230357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2f6c1-d9d1-4cc3-991b-8fb97f4696ee None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:09.231067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2f6c1-d9d1-4cc3-991b-8fb97f4696ee None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:09.235331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 170/337] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:09.242640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51626700-ac78-4e46-98ae-b5231ce263bf None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:09.243063 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51626700-ac78-4e46-98ae-b5231ce263bf None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:09.243432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51626700-ac78-4e46-98ae-b5231ce263bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:09.282446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-51626700-ac78-4e46-98ae-b5231ce263bf None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:09.284762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-51626700-ac78-4e46-98ae-b5231ce263bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:09.305866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:09.426907 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8174bc-b514-4274-ba0c-5f52379014a2 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:09.427913 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8174bc-b514-4274-ba0c-5f52379014a2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:09.428662 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8174bc-b514-4274-ba0c-5f52379014a2 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:09.436227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 171/338] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:09:09.447828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1b314-9eb1-468a-be15-0a00e3a7d77f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:09.449000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1b314-9eb1-468a-be15-0a00e3a7d77f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:09.449901 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1b314-9eb1-468a-be15-0a00e3a7d77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:09.491775 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 168/339] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:09.500720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0ff1b314-9eb1-468a-be15-0a00e3a7d77f None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:09.507082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0ff1b314-9eb1-468a-be15-0a00e3a7d77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:09.666391 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 172/340] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:09.883700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51626700-ac78-4e46-98ae-b5231ce263bf None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:10.122171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51626700-ac78-4e46-98ae-b5231ce263bf None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:10.178659 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a7e0a-7cfe-408f-aa4e-c3ff43ea2d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:10.179335 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a7e0a-7cfe-408f-aa4e-c3ff43ea2d64 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:10.179607 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a7e0a-7cfe-408f-aa4e-c3ff43ea2d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:10.179891 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a7e0a-7cfe-408f-aa4e-c3ff43ea2d64 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:10.196188 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a7e0a-7cfe-408f-aa4e-c3ff43ea2d64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:10.201125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-5e6a7e0a-7cfe-408f-aa4e-c3ff43ea2d64 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:10.201125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:10.201125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:10.201125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:10.201125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:10.201125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:10.201125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:10.201125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:10.201125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:10.201125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:10.201125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:10.201125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:10.201125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:10.201125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:10.201125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:10.201125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:10.203243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:10.203243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:10.203243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:10.203243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:10.203243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:10.203243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:10.203243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:10.203243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:10.203243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:10.203243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:10.203243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:10.203243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:10.203243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:10.203243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:10.203243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:10.203243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:10.205224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:10.205224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:10.205224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:10.205224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:10.205224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:10.205224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:10.205224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:10.205224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:10.205224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:10.205224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:10.205224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:10.205224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:10.205224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 169/341] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:09:10.211977 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1b314-9eb1-468a-be15-0a00e3a7d77f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:10.293805 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a7e0a-7cfe-408f-aa4e-c3ff43ea2d64 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:10.297558 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:10.447455 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1b314-9eb1-468a-be15-0a00e3a7d77f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:10.500311 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:10.515022 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b598a-2cf7-4eae-822e-c17fd4956dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:10.516088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b598a-2cf7-4eae-822e-c17fd4956dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:10.516575 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b598a-2cf7-4eae-822e-c17fd4956dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:10.517023 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b598a-2cf7-4eae-822e-c17fd4956dd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:10.518330 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b598a-2cf7-4eae-822e-c17fd4956dd3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:10.519371 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b598a-2cf7-4eae-822e-c17fd4956dd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:10.521885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b598a-2cf7-4eae-822e-c17fd4956dd3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:10.538995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 173/342] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:10.547771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b598a-2cf7-4eae-822e-c17fd4956dd3 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:10.550267 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a7e0a-7cfe-408f-aa4e-c3ff43ea2d64 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:10.649956 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca5eda-3ec9-4769-addb-f5b0f0ef6aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:10.650634 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca5eda-3ec9-4769-addb-f5b0f0ef6aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:10.650918 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca5eda-3ec9-4769-addb-f5b0f0ef6aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:10.651189 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca5eda-3ec9-4769-addb-f5b0f0ef6aa6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:10.679762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca5eda-3ec9-4769-addb-f5b0f0ef6aa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:10.689220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-d6ca5eda-3ec9-4769-addb-f5b0f0ef6aa6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:10.689220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:10.689220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:10.689220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:10.689220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:10.689220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:10.689220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:10.689220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:10.689220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:10.689220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:10.689220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:10.689220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:10.689220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:10.689220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:10.689220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:10.689220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:10.691340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:10.691340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:10.691340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:10.691340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:10.691340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:10.691340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:10.691340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:10.691340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:10.691340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:10.691340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:10.691340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:10.691340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:10.691340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:10.691340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:10.691340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:10.691340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:10.692181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:10.692181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:10.692181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:10.692181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:10.692181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:10.692181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:10.692181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:10.692181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:10.692181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:10.692181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:10.692181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:10.692181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:10.697420 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 170/343] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:09:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:10.698916 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca5eda-3ec9-4769-addb-f5b0f0ef6aa6 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:10.791227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b598a-2cf7-4eae-822e-c17fd4956dd3 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:10.880584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:10.898515 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2869c-f50d-43c9-a16d-1a387146bddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:10.899387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2869c-f50d-43c9-a16d-1a387146bddb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:10.899671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2869c-f50d-43c9-a16d-1a387146bddb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:10.899935 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2869c-f50d-43c9-a16d-1a387146bddb None admin] PATH_INFO: `/v3/users/5d722b0c96414362a7c81067e6c6d170` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:10.906088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2869c-f50d-43c9-a16d-1a387146bddb None admin] RBAC: Authorizing `identity:get_user(user_id=5d722b0c96414362a7c81067e6c6d170)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:10.907624 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2869c-f50d-43c9-a16d-1a387146bddb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:10.910432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 174/344] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:10 2022] GET /identity/v3/users/5d722b0c96414362a7c81067e6c6d170 => generated 293 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:10.911361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2869c-f50d-43c9-a16d-1a387146bddb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:10.960185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca5eda-3ec9-4769-addb-f5b0f0ef6aa6 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:11.012235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f83d2-38df-4fa3-afc2-f1b9a43f3d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:11.014016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f83d2-38df-4fa3-afc2-f1b9a43f3d87 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:11.014491 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f83d2-38df-4fa3-afc2-f1b9a43f3d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:11.015114 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f83d2-38df-4fa3-afc2-f1b9a43f3d87 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:11.017257 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:11.057718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f83d2-38df-4fa3-afc2-f1b9a43f3d87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:11.067565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-b18f83d2-38df-4fa3-afc2-f1b9a43f3d87 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:11.067565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:11.067565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:11.067565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:11.067565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:11.067565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:11.067565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:11.067565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:11.067565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:11.067565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:11.067565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:11.067565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:11.067565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:11.067565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:11.067565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:11.067565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:11.068647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:11.068647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:11.068647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:11.068647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:11.068647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:11.068647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:11.068647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:11.068647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:11.068647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:11.068647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:11.068647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:11.068647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:11.068647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:11.068647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:11.068647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:11.068647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:11.069655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:11.069655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:11.069655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:11.069655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:11.069655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:11.069655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:11.069655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:11.069655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:11.069655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:11.069655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:11.069655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:11.069655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:11.071551 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 171/345] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:11.072890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f83d2-38df-4fa3-afc2-f1b9a43f3d87 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:11.097680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2869c-f50d-43c9-a16d-1a387146bddb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:11.159819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661ff39-fb0d-4986-9a12-6688f1c084a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:11.160631 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661ff39-fb0d-4986-9a12-6688f1c084a3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:11.160913 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661ff39-fb0d-4986-9a12-6688f1c084a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:11.161232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661ff39-fb0d-4986-9a12-6688f1c084a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:11.161942 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661ff39-fb0d-4986-9a12-6688f1c084a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:11.162246 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661ff39-fb0d-4986-9a12-6688f1c084a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:11.163828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661ff39-fb0d-4986-9a12-6688f1c084a3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:11.176064 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 175/346] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:11.177669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e3212-7e4e-4f24-9582-925fce081e30 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:11.177956 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e3212-7e4e-4f24-9582-925fce081e30 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:11.178228 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e3212-7e4e-4f24-9582-925fce081e30 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:11.180476 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 176/347] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:11.181718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e3212-7e4e-4f24-9582-925fce081e30 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:11.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f83d2-38df-4fa3-afc2-f1b9a43f3d87 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:11.227753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:11.256927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bb6c0-c013-4b78-b8d0-921de44454c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:11.258032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bb6c0-c013-4b78-b8d0-921de44454c4 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:11.258522 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bb6c0-c013-4b78-b8d0-921de44454c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:11.259005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bb6c0-c013-4b78-b8d0-921de44454c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:11.260402 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bb6c0-c013-4b78-b8d0-921de44454c4 None admin] RBAC: Adding query filter params (role.id=8d5b36826f914e958bd80fc21156e286, user.id=5d722b0c96414362a7c81067e6c6d170, scope.system=all) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:11.260945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bb6c0-c013-4b78-b8d0-921de44454c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:11.263787 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bb6c0-c013-4b78-b8d0-921de44454c4 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:11.290928 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 172/348] 10.222.0.88 () {56 vars in 1308 bytes} [Tue Aug 23 22:09:11 2022] GET /identity/v3/role_assignments?user.id=5d722b0c96414362a7c81067e6c6d170&scope.system=all&role.id=8d5b36826f914e958bd80fc21156e286 => generated 510 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:11.292398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bb6c0-c013-4b78-b8d0-921de44454c4 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:11.439884 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e3212-7e4e-4f24-9582-925fce081e30 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:11.519076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:11.570185 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff02e0-430d-4a06-8180-34f375774652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:11.571195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff02e0-430d-4a06-8180-34f375774652 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:11.571648 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff02e0-430d-4a06-8180-34f375774652 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:11.572084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff02e0-430d-4a06-8180-34f375774652 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:11.573337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff02e0-430d-4a06-8180-34f375774652 None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:11.573807 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff02e0-430d-4a06-8180-34f375774652 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:11.576314 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff02e0-430d-4a06-8180-34f375774652 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:11.601593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 177/349] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:11.603823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b91e9-df28-4100-aea5-de5a807e403e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:11.604292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b91e9-df28-4100-aea5-de5a807e403e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:11.604730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b91e9-df28-4100-aea5-de5a807e403e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:11.604857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bb6c0-c013-4b78-b8d0-921de44454c4 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:11.655044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c60b91e9-df28-4100-aea5-de5a807e403e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:11.657130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c60b91e9-df28-4100-aea5-de5a807e403e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:11.675858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4832573-26a1-49ee-b9b1-26559607b23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:11.676704 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832573-26a1-49ee-b9b1-26559607b23a None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:11.677254 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832573-26a1-49ee-b9b1-26559607b23a None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:11.677711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832573-26a1-49ee-b9b1-26559607b23a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:11.779783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:11.827613 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4832573-26a1-49ee-b9b1-26559607b23a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:11.829173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4832573-26a1-49ee-b9b1-26559607b23a None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:11.834540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 178/350] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:11.835427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b91e9-df28-4100-aea5-de5a807e403e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:12.009747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b91e9-df28-4100-aea5-de5a807e403e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:12.015518 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 173/351] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 724 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:12.022439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3168b8-c4da-4d4c-9f42-1099a646daec None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:12.022911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3168b8-c4da-4d4c-9f42-1099a646daec None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:12.023250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3168b8-c4da-4d4c-9f42-1099a646daec None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:12.026673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 174/352] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:12.031638 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:12.033000 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3168b8-c4da-4d4c-9f42-1099a646daec None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:12.073731 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca3b88-9a0e-464d-b7fe-da647ed3fb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:12.074540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca3b88-9a0e-464d-b7fe-da647ed3fb56 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:12.074831 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca3b88-9a0e-464d-b7fe-da647ed3fb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:12.075174 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca3b88-9a0e-464d-b7fe-da647ed3fb56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:12.185243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:12.194516 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca3b88-9a0e-464d-b7fe-da647ed3fb56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:12.196032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca3b88-9a0e-464d-b7fe-da647ed3fb56 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:12.216219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3168b8-c4da-4d4c-9f42-1099a646daec None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:12.266213 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f6f89-2bcf-4666-8638-4cfebbf24a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:12.267674 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f6f89-2bcf-4666-8638-4cfebbf24a1b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:12.268198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f6f89-2bcf-4666-8638-4cfebbf24a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:12.268464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f6f89-2bcf-4666-8638-4cfebbf24a1b None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:12.269795 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f6f89-2bcf-4666-8638-4cfebbf24a1b None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:12.271263 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f6f89-2bcf-4666-8638-4cfebbf24a1b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:12.273277 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 175/353] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:12 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:12.274687 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3f36a-7ed2-4088-b9a7-f562dccafe77 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:12.274957 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3f36a-7ed2-4088-b9a7-f562dccafe77 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:12.275228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3f36a-7ed2-4088-b9a7-f562dccafe77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:12.297845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-acc3f36a-7ed2-4088-b9a7-f562dccafe77 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:12.299367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-acc3f36a-7ed2-4088-b9a7-f562dccafe77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:12.343595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 179/354] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:12.345824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f522cf8-1324-4f73-9d20-86fc139ed95d None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:12.346303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f522cf8-1324-4f73-9d20-86fc139ed95d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:12.347296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f522cf8-1324-4f73-9d20-86fc139ed95d None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:12.352982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 180/355] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:12.354445 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f522cf8-1324-4f73-9d20-86fc139ed95d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:12.383776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:12.403784 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 176/356] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:12.406344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3f36a-7ed2-4088-b9a7-f562dccafe77 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:12.544725 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f522cf8-1324-4f73-9d20-86fc139ed95d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:12.556010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:12.590319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e8512-d44d-4744-8199-6d7d3ad62d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:12.591512 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e8512-d44d-4744-8199-6d7d3ad62d88 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:12.592035 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e8512-d44d-4744-8199-6d7d3ad62d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:12.592788 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e8512-d44d-4744-8199-6d7d3ad62d88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:12.597358 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3f36a-7ed2-4088-b9a7-f562dccafe77 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:12.622839 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e8512-d44d-4744-8199-6d7d3ad62d88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:12.630171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-ea6e8512-d44d-4744-8199-6d7d3ad62d88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:12.630171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:12.630171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:12.630171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:12.630171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:12.630171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:12.630171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:12.630171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:12.630171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:12.630171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:12.630171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:12.630171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:12.630171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:12.630171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:12.630171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:12.630171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:12.631111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:12.631111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:12.631111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:12.631111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:12.631111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:12.631111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:12.631111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:12.631111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:12.631111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:12.631111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:12.631111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:12.631111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:12.631111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:12.631111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:12.631111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:12.631111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:12.631898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:12.631898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:12.631898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:12.631898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:12.631898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:12.631898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:12.631898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:12.631898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:12.631898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:12.631898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:12.631898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:12.631898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:12.631898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 181/357] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:12.632769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e8512-d44d-4744-8199-6d7d3ad62d88 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:12.663656 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa49f10-c84e-4f76-9b8c-50e94e12573f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:12.664318 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa49f10-c84e-4f76-9b8c-50e94e12573f None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:12.664607 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa49f10-c84e-4f76-9b8c-50e94e12573f None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:12.664854 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa49f10-c84e-4f76-9b8c-50e94e12573f None admin] PATH_INFO: `/v3/groups/c6f84987009b42c4b25b0f33152fa58a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:12.679435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa49f10-c84e-4f76-9b8c-50e94e12573f None admin] RBAC: Authorizing `identity:get_group(group_id=c6f84987009b42c4b25b0f33152fa58a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:12.683554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa49f10-c84e-4f76-9b8c-50e94e12573f None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:12.688071 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 177/358] 10.222.0.88 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:12 2022] GET /identity/v3/groups/c6f84987009b42c4b25b0f33152fa58a => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:12.688071 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a4b98-7c44-4726-b4aa-91e56973d030 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:12.688420 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a4b98-7c44-4726-b4aa-91e56973d030 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:12.688545 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a4b98-7c44-4726-b4aa-91e56973d030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:12.725164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-9f3a4b98-7c44-4726-b4aa-91e56973d030 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:12.727792 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-9f3a4b98-7c44-4726-b4aa-91e56973d030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:12.770643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:12.820581 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e8512-d44d-4744-8199-6d7d3ad62d88 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:12.856265 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 178/359] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:12.857875 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a4b98-7c44-4726-b4aa-91e56973d030 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:12.867467 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea8505-e51b-4e14-8f42-b1cc61c0f66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:12.869428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea8505-e51b-4e14-8f42-b1cc61c0f66e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:12.869935 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea8505-e51b-4e14-8f42-b1cc61c0f66e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:12.870438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea8505-e51b-4e14-8f42-b1cc61c0f66e None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:12.873291 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea8505-e51b-4e14-8f42-b1cc61c0f66e None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:12.875502 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea8505-e51b-4e14-8f42-b1cc61c0f66e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:12.880002 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 182/360] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:12 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:12.881190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea8505-e51b-4e14-8f42-b1cc61c0f66e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:12.997821 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:13.070884 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a4b98-7c44-4726-b4aa-91e56973d030 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:13.095542 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea8505-e51b-4e14-8f42-b1cc61c0f66e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:13.148003 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fb968-1f32-4ffb-a37e-26862b2b946e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:13.148649 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fb968-1f32-4ffb-a37e-26862b2b946e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:13.148934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fb968-1f32-4ffb-a37e-26862b2b946e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:13.149317 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fb968-1f32-4ffb-a37e-26862b2b946e None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:13.149963 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40fb968-1f32-4ffb-a37e-26862b2b946e None admin] RBAC: Adding query filter params (name=member) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:13.150272 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40fb968-1f32-4ffb-a37e-26862b2b946e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:13.152379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40fb968-1f32-4ffb-a37e-26862b2b946e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:13.160562 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d92c5-e730-4ee5-a896-e0c205f3f71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:13.161573 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d92c5-e730-4ee5-a896-e0c205f3f71b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:13.162808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d92c5-e730-4ee5-a896-e0c205f3f71b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:13.164013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d92c5-e730-4ee5-a896-e0c205f3f71b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:13.164528 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d92c5-e730-4ee5-a896-e0c205f3f71b None admin] RBAC: Adding query filter params (group.id=c6f84987009b42c4b25b0f33152fa58a, role.id=3c32f171f34345e4958d31e3940092d6, scope.project.id=709dbd6287a240f3b6044bf6d6e63334) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:13.165047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d92c5-e730-4ee5-a896-e0c205f3f71b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:13.165942 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 179/361] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:13.167747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d92c5-e730-4ee5-a896-e0c205f3f71b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:13.167955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fb968-1f32-4ffb-a37e-26862b2b946e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:13.181016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 183/362] 10.222.0.88 () {56 vars in 1376 bytes} [Tue Aug 23 22:09:12 2022] GET /identity/v3/role_assignments?group.id=c6f84987009b42c4b25b0f33152fa58a&scope.project.id=709dbd6287a240f3b6044bf6d6e63334&role.id=3c32f171f34345e4958d31e3940092d6 => generated 260 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:13.182548 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d92c5-e730-4ee5-a896-e0c205f3f71b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:13.189881 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:13.348100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fb968-1f32-4ffb-a37e-26862b2b946e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:13.403451 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d92c5-e730-4ee5-a896-e0c205f3f71b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:13.403711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a1b14-3534-4559-a33b-d75351f1ae95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:13.404276 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a1b14-3534-4559-a33b-d75351f1ae95 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:13.404581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a1b14-3534-4559-a33b-d75351f1ae95 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:13.404932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a1b14-3534-4559-a33b-d75351f1ae95 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:13.409550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:13.429527 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a1b14-3534-4559-a33b-d75351f1ae95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:13.437722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-a13a1b14-3534-4559-a33b-d75351f1ae95 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:13.437722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:13.437722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:13.437722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:13.437722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:13.437722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:13.437722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:13.437722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:13.437722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:13.437722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:13.437722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:13.437722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:13.437722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:13.437722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:13.437722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:13.437722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:13.438870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:13.438870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:13.438870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:13.438870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:13.438870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:13.438870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:13.438870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:13.438870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:13.438870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:13.438870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:13.438870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:13.438870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:13.438870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:13.438870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:13.438870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:13.438870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:13.440235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:13.440235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:13.440235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:13.440235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:13.440235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:13.440235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:13.440235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:13.440235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:13.440235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:13.440235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:13.440235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:13.440235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:13.442690 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 180/363] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:09:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:13.443873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a1b14-3534-4559-a33b-d75351f1ae95 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:13.487685 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d6150-79d1-4f44-b0a3-cd28212803f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:13.488291 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d6150-79d1-4f44-b0a3-cd28212803f4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:13.488564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d6150-79d1-4f44-b0a3-cd28212803f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:13.488838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d6150-79d1-4f44-b0a3-cd28212803f4 None admin] PATH_INFO: `/v3/groups/c6f84987009b42c4b25b0f33152fa58a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:13.490217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d6150-79d1-4f44-b0a3-cd28212803f4 None admin] RBAC: Authorizing `identity:get_group(group_id=c6f84987009b42c4b25b0f33152fa58a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:13.491695 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d6150-79d1-4f44-b0a3-cd28212803f4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:13.498287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 184/364] 10.222.0.88 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:13 2022] GET /identity/v3/groups/c6f84987009b42c4b25b0f33152fa58a => generated 226 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:13.498287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d6150-79d1-4f44-b0a3-cd28212803f4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:13.634565 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:13.637615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a1b14-3534-4559-a33b-d75351f1ae95 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:13.697852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853c357-543f-483e-b2c9-3fad148d9e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:13.698912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853c357-543f-483e-b2c9-3fad148d9e88 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:13.699203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853c357-543f-483e-b2c9-3fad148d9e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:13.699487 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853c357-543f-483e-b2c9-3fad148d9e88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:13.706802 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d6150-79d1-4f44-b0a3-cd28212803f4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:13.713889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853c357-543f-483e-b2c9-3fad148d9e88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:13.719187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-b853c357-543f-483e-b2c9-3fad148d9e88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:13.719187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:13.719187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:13.719187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:13.719187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:13.719187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:13.719187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:13.719187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:13.719187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:13.719187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:13.719187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:13.719187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:13.719187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:13.719187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:13.719187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:13.719187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:13.720943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:13.720943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:13.720943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:13.720943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:13.720943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:13.720943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:13.720943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:13.720943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:13.720943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:13.720943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:13.720943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:13.720943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:13.720943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:13.720943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:13.720943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:13.720943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:13.723542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:13.723542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:13.723542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:13.723542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:13.723542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:13.723542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:13.723542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:13.723542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:13.723542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:13.723542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:13.723542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:13.723542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:13.723542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 181/365] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:09:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:13.723542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853c357-543f-483e-b2c9-3fad148d9e88 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:13.771974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9da40-5d9c-4a7b-8df5-12833c58496a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:13.772998 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9da40-5d9c-4a7b-8df5-12833c58496a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:13.773596 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9da40-5d9c-4a7b-8df5-12833c58496a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:13.774012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9da40-5d9c-4a7b-8df5-12833c58496a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:13.850582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:13.918490 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9da40-5d9c-4a7b-8df5-12833c58496a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:13.919992 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9da40-5d9c-4a7b-8df5-12833c58496a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:13.950631 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853c357-543f-483e-b2c9-3fad148d9e88 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:14.006287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21daf7-1c01-487a-8ccb-d8fcb368ef6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:14.007479 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21daf7-1c01-487a-8ccb-d8fcb368ef6d None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:14.007793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21daf7-1c01-487a-8ccb-d8fcb368ef6d None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:14.008073 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21daf7-1c01-487a-8ccb-d8fcb368ef6d None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:14.011957 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21daf7-1c01-487a-8ccb-d8fcb368ef6d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:14.011957 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21daf7-1c01-487a-8ccb-d8fcb368ef6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:14.011957 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21daf7-1c01-487a-8ccb-d8fcb368ef6d None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:14.026023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 182/366] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:14.029562 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21daf7-1c01-487a-8ccb-d8fcb368ef6d None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:14.043760 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:14.090536 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 185/367] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:14.091405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9da40-5d9c-4a7b-8df5-12833c58496a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:14.219282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21daf7-1c01-487a-8ccb-d8fcb368ef6d None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:14.232054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9da40-5d9c-4a7b-8df5-12833c58496a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:14.248414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:14.285987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ce46d-72da-424d-9c98-1d161eda5cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:14.286960 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ce46d-72da-424d-9c98-1d161eda5cfe None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:14.287443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ce46d-72da-424d-9c98-1d161eda5cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:14.287875 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ce46d-72da-424d-9c98-1d161eda5cfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:14.289129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ce46d-72da-424d-9c98-1d161eda5cfe None admin] RBAC: Adding query filter params (group.id=c6f84987009b42c4b25b0f33152fa58a, role.id=fad7ba55962b40459b40dcc2bad395b2, scope.project.id=709dbd6287a240f3b6044bf6d6e63334) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:14.289612 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ce46d-72da-424d-9c98-1d161eda5cfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:14.290147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2029e7-1dc0-4741-9fe1-c58a52888ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:14.290781 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2029e7-1dc0-4741-9fe1-c58a52888ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:14.291070 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2029e7-1dc0-4741-9fe1-c58a52888ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:14.291343 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2029e7-1dc0-4741-9fe1-c58a52888ec5 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:14.292043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2029e7-1dc0-4741-9fe1-c58a52888ec5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:14.292193 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ce46d-72da-424d-9c98-1d161eda5cfe None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:14.292338 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2029e7-1dc0-4741-9fe1-c58a52888ec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:14.293794 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2029e7-1dc0-4741-9fe1-c58a52888ec5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:14.304693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 186/368] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:14.309376 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 183/369] 10.222.0.88 () {56 vars in 1376 bytes} [Tue Aug 23 22:09:14 2022] GET /identity/v3/role_assignments?group.id=c6f84987009b42c4b25b0f33152fa58a&scope.project.id=709dbd6287a240f3b6044bf6d6e63334&role.id=fad7ba55962b40459b40dcc2bad395b2 => generated 260 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:14.315815 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ce46d-72da-424d-9c98-1d161eda5cfe None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:14.316095 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2029e7-1dc0-4741-9fe1-c58a52888ec5 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:14.395579 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:14.524382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2029e7-1dc0-4741-9fe1-c58a52888ec5 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:14.542175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ce46d-72da-424d-9c98-1d161eda5cfe None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:14.585937 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25318b20-d99a-45ab-a9b8-d2d378310829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:14.587218 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25318b20-d99a-45ab-a9b8-d2d378310829 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:14.587658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25318b20-d99a-45ab-a9b8-d2d378310829 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:14.588111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25318b20-d99a-45ab-a9b8-d2d378310829 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:14.604340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13379e-eec9-486d-a4fa-4380baa1b74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:14.605570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13379e-eec9-486d-a4fa-4380baa1b74e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:14.606104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13379e-eec9-486d-a4fa-4380baa1b74e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:14.606647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13379e-eec9-486d-a4fa-4380baa1b74e None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:14.609569 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13379e-eec9-486d-a4fa-4380baa1b74e None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:14.613475 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13379e-eec9-486d-a4fa-4380baa1b74e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:14.621191 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 184/370] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:14 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:14.623030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13379e-eec9-486d-a4fa-4380baa1b74e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:14.718894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:14.743828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25318b20-d99a-45ab-a9b8-d2d378310829 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:14.746919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25318b20-d99a-45ab-a9b8-d2d378310829 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:14.788077 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13379e-eec9-486d-a4fa-4380baa1b74e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:14.818844 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:14.834541 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28551af4-c815-4f09-951f-2ef75e9e54ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:14.835187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28551af4-c815-4f09-951f-2ef75e9e54ae None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:14.835469 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28551af4-c815-4f09-951f-2ef75e9e54ae None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:14.835744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28551af4-c815-4f09-951f-2ef75e9e54ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:14.921694 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28551af4-c815-4f09-951f-2ef75e9e54ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:14.923231 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28551af4-c815-4f09-951f-2ef75e9e54ae None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:14.974018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 187/371] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:14.975833 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25318b20-d99a-45ab-a9b8-d2d378310829 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:14.984441 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:15.082253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 185/372] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:15.083303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28551af4-c815-4f09-951f-2ef75e9e54ae None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:15.197523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25318b20-d99a-45ab-a9b8-d2d378310829 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:15.277447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639606c-9198-4b12-b6f0-2667b8a54c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:15.279053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639606c-9198-4b12-b6f0-2667b8a54c07 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:15.279758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639606c-9198-4b12-b6f0-2667b8a54c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:15.280461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639606c-9198-4b12-b6f0-2667b8a54c07 None admin] PATH_INFO: `/v3/groups/c6f84987009b42c4b25b0f33152fa58a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:15.283244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1639606c-9198-4b12-b6f0-2667b8a54c07 None admin] RBAC: Authorizing `identity:get_group(group_id=c6f84987009b42c4b25b0f33152fa58a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:15.286733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1639606c-9198-4b12-b6f0-2667b8a54c07 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:15.297905 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 188/373] 10.222.0.88 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:14 2022] GET /identity/v3/groups/c6f84987009b42c4b25b0f33152fa58a => generated 226 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:15.300483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639606c-9198-4b12-b6f0-2667b8a54c07 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:15.306955 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:15.314038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28551af4-c815-4f09-951f-2ef75e9e54ae None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:15.404076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d3bc3-acc0-4b6e-80eb-7100c346e170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:15.405319 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d3bc3-acc0-4b6e-80eb-7100c346e170 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:15.405854 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d3bc3-acc0-4b6e-80eb-7100c346e170 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:15.406433 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d3bc3-acc0-4b6e-80eb-7100c346e170 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:15.410736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d3bc3-acc0-4b6e-80eb-7100c346e170 None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:15.414313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d3bc3-acc0-4b6e-80eb-7100c346e170 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:15.419340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 186/374] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:15 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:15.420927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d3bc3-acc0-4b6e-80eb-7100c346e170 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:15.454646 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:15.516128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639606c-9198-4b12-b6f0-2667b8a54c07 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:15.588750 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d8971-a47b-4497-9ddb-839d127ddd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:15.590410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d8971-a47b-4497-9ddb-839d127ddd55 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:15.591160 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d8971-a47b-4497-9ddb-839d127ddd55 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:15.591846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d8971-a47b-4497-9ddb-839d127ddd55 None admin] PATH_INFO: `/v3/projects/fe428a23adc74bbab88ff935243cc4e2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:15.594409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d8971-a47b-4497-9ddb-839d127ddd55 None admin] RBAC: Authorizing `identity:get_project(project_id=fe428a23adc74bbab88ff935243cc4e2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:15.598514 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d8971-a47b-4497-9ddb-839d127ddd55 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:15.602759 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 189/375] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:15 2022] GET /identity/v3/projects/fe428a23adc74bbab88ff935243cc4e2 => generated 344 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:15.604687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d8971-a47b-4497-9ddb-839d127ddd55 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:15.652616 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d3bc3-acc0-4b6e-80eb-7100c346e170 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:15.719509 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8f196-f84e-4a9b-8172-dcfb61df1f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:15.720145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8f196-f84e-4a9b-8172-dcfb61df1f68 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:15.720502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8f196-f84e-4a9b-8172-dcfb61df1f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:15.720874 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8f196-f84e-4a9b-8172-dcfb61df1f68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:15.722661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8f196-f84e-4a9b-8172-dcfb61df1f68 None admin] RBAC: Adding query filter params (group.id=c6f84987009b42c4b25b0f33152fa58a, role.id=fad7ba55962b40459b40dcc2bad395b2, scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:15.723059 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8f196-f84e-4a9b-8172-dcfb61df1f68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:15.724641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8f196-f84e-4a9b-8172-dcfb61df1f68 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:15.735894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 187/376] 10.222.0.88 () {56 vars in 1376 bytes} [Tue Aug 23 22:09:15 2022] GET /identity/v3/role_assignments?group.id=c6f84987009b42c4b25b0f33152fa58a&scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a&role.id=fad7ba55962b40459b40dcc2bad395b2 => generated 260 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:15.738182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8f196-f84e-4a9b-8172-dcfb61df1f68 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:15.750799 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:15.890023 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d8971-a47b-4497-9ddb-839d127ddd55 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:16.015385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08191af9-8a32-4566-adc8-dbecffd9a10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:16.016794 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08191af9-8a32-4566-adc8-dbecffd9a10b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:16.017526 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08191af9-8a32-4566-adc8-dbecffd9a10b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:16.017672 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:16.018903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08191af9-8a32-4566-adc8-dbecffd9a10b None admin] PATH_INFO: `/v3/groups/c6f84987009b42c4b25b0f33152fa58a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.021342 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08191af9-8a32-4566-adc8-dbecffd9a10b None admin] RBAC: Authorizing `identity:get_group(group_id=c6f84987009b42c4b25b0f33152fa58a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:16.024375 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08191af9-8a32-4566-adc8-dbecffd9a10b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:16.028053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 190/377] 10.222.0.88 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:15 2022] GET /identity/v3/groups/c6f84987009b42c4b25b0f33152fa58a => generated 226 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:16.030600 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e05236-2167-4e41-a57b-3767a14f6041 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:16.031415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e05236-2167-4e41-a57b-3767a14f6041 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:16.032408 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e05236-2167-4e41-a57b-3767a14f6041 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.036457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 191/378] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:16.038542 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e05236-2167-4e41-a57b-3767a14f6041 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:16.039118 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8f196-f84e-4a9b-8172-dcfb61df1f68 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:16.084611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cec951-b60a-4c0a-9e85-395727d06fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:16.085956 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cec951-b60a-4c0a-9e85-395727d06fac None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:16.086241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cec951-b60a-4c0a-9e85-395727d06fac None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:16.086518 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cec951-b60a-4c0a-9e85-395727d06fac None admin] PATH_INFO: `/v3/groups/8e985ee66ebd4ebfb87262d34bfec41f` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.093342 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cec951-b60a-4c0a-9e85-395727d06fac None admin] RBAC: Authorizing `identity:get_group(group_id=8e985ee66ebd4ebfb87262d34bfec41f)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:16.095251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cec951-b60a-4c0a-9e85-395727d06fac None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:16.098159 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 188/379] 10.222.0.88 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:15 2022] GET /identity/v3/groups/8e985ee66ebd4ebfb87262d34bfec41f => generated 229 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:16.101402 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47867a7a-2f4f-48bf-89c6-74cb2f5e0a6d None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:16.101934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47867a7a-2f4f-48bf-89c6-74cb2f5e0a6d None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:16.102486 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47867a7a-2f4f-48bf-89c6-74cb2f5e0a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.132215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-47867a7a-2f4f-48bf-89c6-74cb2f5e0a6d None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:16.133548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-47867a7a-2f4f-48bf-89c6-74cb2f5e0a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:16.229911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 189/380] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:16.230792 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47867a7a-2f4f-48bf-89c6-74cb2f5e0a6d None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:16.294297 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e05236-2167-4e41-a57b-3767a14f6041 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:16.321412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:16.413386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47867a7a-2f4f-48bf-89c6-74cb2f5e0a6d None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:16.419515 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccda650-3db9-4d6a-b4cc-f30bdc32dba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:16.420546 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccda650-3db9-4d6a-b4cc-f30bdc32dba0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:16.421031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccda650-3db9-4d6a-b4cc-f30bdc32dba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:16.421529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccda650-3db9-4d6a-b4cc-f30bdc32dba0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.425252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccda650-3db9-4d6a-b4cc-f30bdc32dba0 None admin] RBAC: Adding query filter params (group.id=c6f84987009b42c4b25b0f33152fa58a, role.id=3c32f171f34345e4958d31e3940092d6, scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:16.425809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccda650-3db9-4d6a-b4cc-f30bdc32dba0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:16.435032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccda650-3db9-4d6a-b4cc-f30bdc32dba0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:16.457096 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 192/381] 10.222.0.88 () {56 vars in 1376 bytes} [Tue Aug 23 22:09:16 2022] GET /identity/v3/role_assignments?group.id=c6f84987009b42c4b25b0f33152fa58a&scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a&role.id=3c32f171f34345e4958d31e3940092d6 => generated 260 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:16.511839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9e588-b1a4-47af-bebf-14a54da437a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:16.512974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9e588-b1a4-47af-bebf-14a54da437a0 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:16.513526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9e588-b1a4-47af-bebf-14a54da437a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:16.514078 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9e588-b1a4-47af-bebf-14a54da437a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.515494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9e588-b1a4-47af-bebf-14a54da437a0 None admin] RBAC: Adding query filter params (group.id=8e985ee66ebd4ebfb87262d34bfec41f, role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=fe428a23adc74bbab88ff935243cc4e2) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:16.516004 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9e588-b1a4-47af-bebf-14a54da437a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:16.520005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9e588-b1a4-47af-bebf-14a54da437a0 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:16.530303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 190/382] 10.222.0.88 () {56 vars in 1376 bytes} [Tue Aug 23 22:09:16 2022] GET /identity/v3/role_assignments?group.id=8e985ee66ebd4ebfb87262d34bfec41f&scope.project.id=fe428a23adc74bbab88ff935243cc4e2&role.id=dc5d71ab685e420697f520aa3a368177 => generated 260 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:16.872444 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccda650-3db9-4d6a-b4cc-f30bdc32dba0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:16.877300 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:17.117633 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccda650-3db9-4d6a-b4cc-f30bdc32dba0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:17.167552 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74442ae-3471-4405-a231-0feb40ffae4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:17.168186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74442ae-3471-4405-a231-0feb40ffae4f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:17.168465 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74442ae-3471-4405-a231-0feb40ffae4f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:17.168737 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74442ae-3471-4405-a231-0feb40ffae4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:17.190444 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74442ae-3471-4405-a231-0feb40ffae4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:17.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-a74442ae-3471-4405-a231-0feb40ffae4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:17.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:17.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:17.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:17.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:17.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:17.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:17.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:17.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:17.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:17.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:17.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:17.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:17.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:17.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:17.196819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:17.198518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:17.198518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:17.198518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:17.198518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:17.198518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:17.198518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:17.198518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:17.198518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:17.198518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:17.198518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:17.198518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:17.198518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:17.198518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:17.198518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:17.198518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:17.198518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:17.200664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:17.200664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:17.200664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:17.200664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:17.200664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:17.200664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:17.200664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:17.200664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:17.200664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:17.200664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:17.200664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:17.200664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:17.200664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 193/383] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:09:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:17.210924 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9e588-b1a4-47af-bebf-14a54da437a0 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:17.380172 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9e588-b1a4-47af-bebf-14a54da437a0 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:17.418564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cb301-7143-40a1-b303-7dd7b55357f4 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:17.418819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cb301-7143-40a1-b303-7dd7b55357f4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:17.419100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cb301-7143-40a1-b303-7dd7b55357f4 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:17.420471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 194/384] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:09:17.435867 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44aad93-255e-43a9-8053-0309225566fa None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:17.436145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44aad93-255e-43a9-8053-0309225566fa None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:17.436402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44aad93-255e-43a9-8053-0309225566fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:17.476308 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82eb57-1b41-46e7-b881-e8975baa6aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:17.477734 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82eb57-1b41-46e7-b881-e8975baa6aae None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:17.478446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82eb57-1b41-46e7-b881-e8975baa6aae None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:17.479108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82eb57-1b41-46e7-b881-e8975baa6aae None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:17.480504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82eb57-1b41-46e7-b881-e8975baa6aae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:17.481310 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82eb57-1b41-46e7-b881-e8975baa6aae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:17.483749 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82eb57-1b41-46e7-b881-e8975baa6aae None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:17.486923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b44aad93-255e-43a9-8053-0309225566fa None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:17.493805 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b44aad93-255e-43a9-8053-0309225566fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:17.494670 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:17.510940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 191/385] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:17.519847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82eb57-1b41-46e7-b881-e8975baa6aae None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:17.696869 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:17.749006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 195/386] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:17.781981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82eb57-1b41-46e7-b881-e8975baa6aae None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:17.859139 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643bab9-33b4-4db6-88f5-4b932c556424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:17.862563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643bab9-33b4-4db6-88f5-4b932c556424 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:17.862563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643bab9-33b4-4db6-88f5-4b932c556424 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:17.862563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643bab9-33b4-4db6-88f5-4b932c556424 None admin] PATH_INFO: `/v3/groups/c6f84987009b42c4b25b0f33152fa58a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:17.863572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643bab9-33b4-4db6-88f5-4b932c556424 None admin] RBAC: Authorizing `identity:get_group(group_id=c6f84987009b42c4b25b0f33152fa58a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:17.866500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643bab9-33b4-4db6-88f5-4b932c556424 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:17.869798 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 192/387] 10.222.0.88 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:17 2022] GET /identity/v3/groups/c6f84987009b42c4b25b0f33152fa58a => generated 226 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:17.878879 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44aad93-255e-43a9-8053-0309225566fa None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:18.137443 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44aad93-255e-43a9-8053-0309225566fa None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:18.206946 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165c662-207d-44f9-badc-d79bec371869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:18.208586 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165c662-207d-44f9-badc-d79bec371869 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:18.209407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165c662-207d-44f9-badc-d79bec371869 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:18.210189 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165c662-207d-44f9-badc-d79bec371869 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:18.212698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165c662-207d-44f9-badc-d79bec371869 None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:18.215887 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165c662-207d-44f9-badc-d79bec371869 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:18.219962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 196/388] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:17 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:18.228977 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643bab9-33b4-4db6-88f5-4b932c556424 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:18.232994 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165c662-207d-44f9-badc-d79bec371869 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:18.344202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:18.360754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:18.416293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643bab9-33b4-4db6-88f5-4b932c556424 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:18.449678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165c662-207d-44f9-badc-d79bec371869 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:18.508230 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1398a5e-25e3-4fdd-a754-a140b8ba2063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:18.509385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1398a5e-25e3-4fdd-a754-a140b8ba2063 None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:18.509836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1398a5e-25e3-4fdd-a754-a140b8ba2063 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:18.510276 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1398a5e-25e3-4fdd-a754-a140b8ba2063 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334/groups/c6f84987009b42c4b25b0f33152fa58a/roles/3c32f171f34345e4958d31e3940092d6` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:18.513742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1398a5e-25e3-4fdd-a754-a140b8ba2063 None admin] RBAC: Authorizing `identity:create_grant(project_id=709dbd6287a240f3b6044bf6d6e63334, group_id=c6f84987009b42c4b25b0f33152fa58a, role_id=3c32f171f34345e4958d31e3940092d6)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:18.515234 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1398a5e-25e3-4fdd-a754-a140b8ba2063 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:18.527650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b97c0-66c3-4ca8-8d55-36466e42d305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:18.529467 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b97c0-66c3-4ca8-8d55-36466e42d305 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:18.530299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b97c0-66c3-4ca8-8d55-36466e42d305 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:18.531116 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b97c0-66c3-4ca8-8d55-36466e42d305 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:18.556070 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b97c0-66c3-4ca8-8d55-36466e42d305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:18.568520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-9c2b97c0-66c3-4ca8-8d55-36466e42d305 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:18.568520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:18.568520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:18.568520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:18.568520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:18.568520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:18.568520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:18.568520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:18.568520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:18.568520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:18.568520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:18.568520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:18.568520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:18.568520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:18.568520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:18.568520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:18.570447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:18.570447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:18.570447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:18.570447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:18.570447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:18.570447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:18.570447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:18.570447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:18.570447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:18.570447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:18.570447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:18.570447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:18.570447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:18.570447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:18.570447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:18.570447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:18.574488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:18.574488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:18.574488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:18.574488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:18.574488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:18.574488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:18.574488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:18.574488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:18.574488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:18.574488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:18.574488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:18.574488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:18.575791 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 193/389] 10.222.0.88 () {58 vars in 1442 bytes} [Tue Aug 23 22:09:18 2022] PUT /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334/groups/c6f84987009b42c4b25b0f33152fa58a/roles/3c32f171f34345e4958d31e3940092d6 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:18.579902 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 197/390] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:18.591642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1398a5e-25e3-4fdd-a754-a140b8ba2063 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:18.703391 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1ec30-6f2a-4ad3-8563-f7fe46fe8b16 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:18.704353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1ec30-6f2a-4ad3-8563-f7fe46fe8b16 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:18.709437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1ec30-6f2a-4ad3-8563-f7fe46fe8b16 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:18.713042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 198/391] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:18 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:18.719805 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712f9bc-9280-43d5-ab35-7cffdc3afcd3 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:18.720714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712f9bc-9280-43d5-ab35-7cffdc3afcd3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:18.723149 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712f9bc-9280-43d5-ab35-7cffdc3afcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:18.776661 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a712f9bc-9280-43d5-ab35-7cffdc3afcd3 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:18.779042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a712f9bc-9280-43d5-ab35-7cffdc3afcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:18.866668 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1398a5e-25e3-4fdd-a754-a140b8ba2063 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:18.892447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 199/392] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:18.905089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:18.916735 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291637c3-cc3f-4893-8f61-b45862958d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:18.917405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291637c3-cc3f-4893-8f61-b45862958d5f None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:18.918592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291637c3-cc3f-4893-8f61-b45862958d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:18.918881 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291637c3-cc3f-4893-8f61-b45862958d5f None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:18.919626 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291637c3-cc3f-4893-8f61-b45862958d5f None admin] RBAC: Adding query filter params (name=member) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:18.919940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291637c3-cc3f-4893-8f61-b45862958d5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:18.921395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291637c3-cc3f-4893-8f61-b45862958d5f None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:18.932208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 194/393] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:18.937850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712f9bc-9280-43d5-ab35-7cffdc3afcd3 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:19.007633 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:19.073920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712f9bc-9280-43d5-ab35-7cffdc3afcd3 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:19.116584 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffaa497-57ae-4834-938a-35b5e8a09693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:19.117264 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffaa497-57ae-4834-938a-35b5e8a09693 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:19.117542 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffaa497-57ae-4834-938a-35b5e8a09693 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:19.117811 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffaa497-57ae-4834-938a-35b5e8a09693 None admin] PATH_INFO: `/v3/groups/c6f84987009b42c4b25b0f33152fa58a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:19.119006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffaa497-57ae-4834-938a-35b5e8a09693 None admin] RBAC: Authorizing `identity:get_group(group_id=c6f84987009b42c4b25b0f33152fa58a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:19.120579 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffaa497-57ae-4834-938a-35b5e8a09693 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:19.122619 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 200/394] 10.222.0.88 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:18 2022] GET /identity/v3/groups/c6f84987009b42c4b25b0f33152fa58a => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:19.129216 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291637c3-cc3f-4893-8f61-b45862958d5f None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:19.287375 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffaa497-57ae-4834-938a-35b5e8a09693 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:19.403502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291637c3-cc3f-4893-8f61-b45862958d5f None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:19.423293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffaa497-57ae-4834-938a-35b5e8a09693 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:19.470028 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:19.498799 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c563b-71e4-488f-b1af-dafdda5648a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:19.499452 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c563b-71e4-488f-b1af-dafdda5648a0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:19.499736 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c563b-71e4-488f-b1af-dafdda5648a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:19.500009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c563b-71e4-488f-b1af-dafdda5648a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:19.507402 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a79da7-6b47-423d-8821-c7e663f42877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:19.508443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a79da7-6b47-423d-8821-c7e663f42877 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:19.508948 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a79da7-6b47-423d-8821-c7e663f42877 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:19.509461 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a79da7-6b47-423d-8821-c7e663f42877 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:19.512542 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c563b-71e4-488f-b1af-dafdda5648a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:19.513639 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a79da7-6b47-423d-8821-c7e663f42877 None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:19.515343 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a79da7-6b47-423d-8821-c7e663f42877 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:19.520409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-0b5c563b-71e4-488f-b1af-dafdda5648a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:19.520409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:19.520409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:19.520409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:19.520409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:19.520409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:19.520409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:19.520409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:19.520409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:19.520409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:19.520409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:19.520409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:19.520409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:19.520409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:19.520409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:19.520409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:19.522244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:19.522244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:19.522244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:19.522244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:19.522244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:19.522244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:19.522244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:19.522244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:19.522244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:19.522244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:19.522244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:19.522244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:19.522244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:19.522244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:19.522244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:19.522244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:19.523306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:19.523306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:19.523306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:19.523306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:19.523306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:19.523306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:19.523306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:19.523306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:19.523306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:19.523306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:19.523306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:19.523306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:19.523306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 201/396] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:19.524926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 195/396] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:19 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:19.529391 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c563b-71e4-488f-b1af-dafdda5648a0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:19.530979 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a79da7-6b47-423d-8821-c7e663f42877 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:19.659604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c563b-71e4-488f-b1af-dafdda5648a0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:19.692545 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:19.703320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e320d-68f9-484b-80ee-6fe8fc461573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:19.703948 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e320d-68f9-484b-80ee-6fe8fc461573 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:19.704224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e320d-68f9-484b-80ee-6fe8fc461573 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:19.704504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e320d-68f9-484b-80ee-6fe8fc461573 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:19.705249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e320d-68f9-484b-80ee-6fe8fc461573 None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:19.706271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e320d-68f9-484b-80ee-6fe8fc461573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:19.707829 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e320d-68f9-484b-80ee-6fe8fc461573 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:19.717870 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 202/397] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:19.724879 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e320d-68f9-484b-80ee-6fe8fc461573 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:19.789474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:19.845531 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a79da7-6b47-423d-8821-c7e663f42877 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:19.905927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d57d7-2775-417b-af5e-eec2ff3986d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:19.906610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d57d7-2775-417b-af5e-eec2ff3986d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:19.906900 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d57d7-2775-417b-af5e-eec2ff3986d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:19.907171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d57d7-2775-417b-af5e-eec2ff3986d3 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334/groups/c6f84987009b42c4b25b0f33152fa58a/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:19.914275 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d57d7-2775-417b-af5e-eec2ff3986d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=709dbd6287a240f3b6044bf6d6e63334, group_id=c6f84987009b42c4b25b0f33152fa58a, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:19.919224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d57d7-2775-417b-af5e-eec2ff3986d3 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:19.957886 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 196/398] 10.222.0.88 () {58 vars in 1442 bytes} [Tue Aug 23 22:09:19 2022] PUT /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334/groups/c6f84987009b42c4b25b0f33152fa58a/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:19.965310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e320d-68f9-484b-80ee-6fe8fc461573 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:20.036076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5aa31f-2dd6-4925-9fcb-f3ee842eb9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:20.037845 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5aa31f-2dd6-4925-9fcb-f3ee842eb9ef None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:20.038668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5aa31f-2dd6-4925-9fcb-f3ee842eb9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:20.039461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5aa31f-2dd6-4925-9fcb-f3ee842eb9ef None admin] PATH_INFO: `/v3/groups/c6f84987009b42c4b25b0f33152fa58a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:20.042167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5aa31f-2dd6-4925-9fcb-f3ee842eb9ef None admin] RBAC: Authorizing `identity:get_group(group_id=c6f84987009b42c4b25b0f33152fa58a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:20.045733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5aa31f-2dd6-4925-9fcb-f3ee842eb9ef None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:20.050778 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 203/399] 10.222.0.88 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:19 2022] GET /identity/v3/groups/c6f84987009b42c4b25b0f33152fa58a => generated 226 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:20.058103 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d57d7-2775-417b-af5e-eec2ff3986d3 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:20.157654 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f8870-7942-4bfd-90f6-791f39a8e21f None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:20.158171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f8870-7942-4bfd-90f6-791f39a8e21f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:20.158704 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f8870-7942-4bfd-90f6-791f39a8e21f None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:20.161903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 204/400] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:09:20.170907 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2bd16-51de-4a00-a970-638cffad2599 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:20.171252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2bd16-51de-4a00-a970-638cffad2599 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:20.171689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2bd16-51de-4a00-a970-638cffad2599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:20.217516 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-19c2bd16-51de-4a00-a970-638cffad2599 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:20.219578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-19c2bd16-51de-4a00-a970-638cffad2599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:20.245663 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d57d7-2775-417b-af5e-eec2ff3986d3 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:20.280574 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:20.316495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b9d44-7f95-4805-adf1-40ad3fe0d188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:20.318279 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b9d44-7f95-4805-adf1-40ad3fe0d188 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:20.318760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b9d44-7f95-4805-adf1-40ad3fe0d188 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:20.319234 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b9d44-7f95-4805-adf1-40ad3fe0d188 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:20.321538 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b9d44-7f95-4805-adf1-40ad3fe0d188 None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:20.323937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b9d44-7f95-4805-adf1-40ad3fe0d188 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:20.326765 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 197/401] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:20 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:20.332685 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b9d44-7f95-4805-adf1-40ad3fe0d188 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:20.353296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 205/402] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:20.380176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e40875-a168-4feb-b276-66dd9e1bfad3 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:20.380460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e40875-a168-4feb-b276-66dd9e1bfad3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:20.380712 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e40875-a168-4feb-b276-66dd9e1bfad3 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:20.385286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 206/403] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:09:20.389256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c45b0-42e1-449d-b81c-c3a464f3bcf1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:20.389905 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c45b0-42e1-449d-b81c-c3a464f3bcf1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:20.390189 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c45b0-42e1-449d-b81c-c3a464f3bcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:20.443594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-122c45b0-42e1-449d-b81c-c3a464f3bcf1 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:20.445581 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-122c45b0-42e1-449d-b81c-c3a464f3bcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:20.531388 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:20.581393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b9d44-7f95-4805-adf1-40ad3fe0d188 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:20.631752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f8cc9-4da6-40a8-abe1-3ccdd040ad82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:20.632964 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f8cc9-4da6-40a8-abe1-3ccdd040ad82 None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:20.633507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f8cc9-4da6-40a8-abe1-3ccdd040ad82 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:20.634000 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f8cc9-4da6-40a8-abe1-3ccdd040ad82 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a/groups/c6f84987009b42c4b25b0f33152fa58a/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:20.637578 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f8cc9-4da6-40a8-abe1-3ccdd040ad82 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc2c956c8a3f4d0f832f927cdc74095a, group_id=c6f84987009b42c4b25b0f33152fa58a, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:20.639728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 207/404] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:20.640430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f8cc9-4da6-40a8-abe1-3ccdd040ad82 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:20.670738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 198/405] 10.222.0.88 () {58 vars in 1442 bytes} [Tue Aug 23 22:09:20 2022] PUT /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a/groups/c6f84987009b42c4b25b0f33152fa58a/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:09:20.922345 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c45b0-42e1-449d-b81c-c3a464f3bcf1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:21.046642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f8cc9-4da6-40a8-abe1-3ccdd040ad82 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:21.116656 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c45b0-42e1-449d-b81c-c3a464f3bcf1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:21.160976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:21.223564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d761142-4e64-4fc5-bc64-ba9d363abffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:21.224237 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d761142-4e64-4fc5-bc64-ba9d363abffe None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:21.224529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d761142-4e64-4fc5-bc64-ba9d363abffe None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:21.224791 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d761142-4e64-4fc5-bc64-ba9d363abffe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:21.268248 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d761142-4e64-4fc5-bc64-ba9d363abffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:21.278177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-3d761142-4e64-4fc5-bc64-ba9d363abffe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:21.278177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:21.278177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:21.278177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:21.278177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:21.278177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:21.278177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:21.278177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:21.278177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:21.278177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:21.278177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:21.278177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:21.278177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:21.278177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:21.278177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:21.278177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:21.279893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:21.279893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:21.279893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:21.279893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:21.279893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:21.279893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:21.279893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:21.279893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:21.279893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:21.279893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:21.279893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:21.279893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:21.279893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:21.279893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:21.279893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:21.279893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:21.281741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:21.281741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:21.281741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:21.281741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:21.281741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:21.281741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:21.281741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:21.281741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:21.281741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:21.281741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:21.281741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:21.281741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:21.294720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 208/406] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:09:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:09:21.304320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d761142-4e64-4fc5-bc64-ba9d363abffe None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:21.318269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f8cc9-4da6-40a8-abe1-3ccdd040ad82 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:21.378382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:21.389403 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfeaa4-3245-4083-92de-82fbb61d2720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:21.390502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfeaa4-3245-4083-92de-82fbb61d2720 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:21.390979 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfeaa4-3245-4083-92de-82fbb61d2720 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:21.391454 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfeaa4-3245-4083-92de-82fbb61d2720 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:21.425029 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfeaa4-3245-4083-92de-82fbb61d2720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:21.441030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-6adfeaa4-3245-4083-92de-82fbb61d2720 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:21.441030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:21.441030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:21.441030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:21.441030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:21.441030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:21.441030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:21.441030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:21.441030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:21.441030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:21.441030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:21.441030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:21.441030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:21.441030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:21.441030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:21.441030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:21.442039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:21.442039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:21.442039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:21.442039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:21.442039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:21.442039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:21.442039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:21.442039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:21.442039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:21.442039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:21.442039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:21.442039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:21.442039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:21.442039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:21.442039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:21.442039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:21.442880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:21.442880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:21.442880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:21.442880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:21.442880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:21.442880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:21.442880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:21.442880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:21.442880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:21.442880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:21.442880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:21.442880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:21.443717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 199/407] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:09:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:21.453142 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b86731-bc5c-45ad-a194-e28b12371fbf None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:21.453725 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b86731-bc5c-45ad-a194-e28b12371fbf None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:21.454192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b86731-bc5c-45ad-a194-e28b12371fbf None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:21.461133 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 200/408] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:21 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:21.461133 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b86731-bc5c-45ad-a194-e28b12371fbf None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:21.497554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d761142-4e64-4fc5-bc64-ba9d363abffe None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:21.573329 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d5fa4-f01c-4a86-a4d1-6003aa9caaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:21.573870 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d5fa4-f01c-4a86-a4d1-6003aa9caaf2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:21.574154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d5fa4-f01c-4a86-a4d1-6003aa9caaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:21.574429 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d5fa4-f01c-4a86-a4d1-6003aa9caaf2 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:21.575108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d5fa4-f01c-4a86-a4d1-6003aa9caaf2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:21.575409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d5fa4-f01c-4a86-a4d1-6003aa9caaf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:21.577595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d5fa4-f01c-4a86-a4d1-6003aa9caaf2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:21.589242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 209/409] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:21.590601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18091ae0-2a6e-4ab2-863b-f1eede003c15 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:21.590877 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18091ae0-2a6e-4ab2-863b-f1eede003c15 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:21.591150 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18091ae0-2a6e-4ab2-863b-f1eede003c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:21.616421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-18091ae0-2a6e-4ab2-863b-f1eede003c15 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:21.617754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-18091ae0-2a6e-4ab2-863b-f1eede003c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:21.634054 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b86731-bc5c-45ad-a194-e28b12371fbf None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:21.650884 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:21.697125 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 210/410] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:21.698065 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18091ae0-2a6e-4ab2-863b-f1eede003c15 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:21.718950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891bf49-d5a4-4b9c-ac4c-b331f735e239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:21.719759 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891bf49-d5a4-4b9c-ac4c-b331f735e239 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:21.720130 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891bf49-d5a4-4b9c-ac4c-b331f735e239 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:21.720506 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891bf49-d5a4-4b9c-ac4c-b331f735e239 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:21.722176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891bf49-d5a4-4b9c-ac4c-b331f735e239 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:21.722333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891bf49-d5a4-4b9c-ac4c-b331f735e239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:21.723989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891bf49-d5a4-4b9c-ac4c-b331f735e239 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:21.726057 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:21.748208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 201/411] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:21.757988 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891bf49-d5a4-4b9c-ac4c-b331f735e239 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:21.916155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18091ae0-2a6e-4ab2-863b-f1eede003c15 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:21.988379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12f662-5c59-4bb6-b36d-0cc7054bc815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:21.989071 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12f662-5c59-4bb6-b36d-0cc7054bc815 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:21.989363 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12f662-5c59-4bb6-b36d-0cc7054bc815 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:21.989639 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12f662-5c59-4bb6-b36d-0cc7054bc815 None admin] PATH_INFO: `/v3/groups/c6f84987009b42c4b25b0f33152fa58a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:21.991510 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12f662-5c59-4bb6-b36d-0cc7054bc815 None admin] RBAC: Authorizing `identity:get_group(group_id=c6f84987009b42c4b25b0f33152fa58a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:21.993340 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12f662-5c59-4bb6-b36d-0cc7054bc815 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:21.995043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 211/412] 10.222.0.88 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:21 2022] GET /identity/v3/groups/c6f84987009b42c4b25b0f33152fa58a => generated 226 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:22.004733 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891bf49-d5a4-4b9c-ac4c-b331f735e239 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:22.006611 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12f662-5c59-4bb6-b36d-0cc7054bc815 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:22.077086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe441e-6ad2-4457-b1e0-e289e186f5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:22.083091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe441e-6ad2-4457-b1e0-e289e186f5dc None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:22.083420 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe441e-6ad2-4457-b1e0-e289e186f5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:22.083637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe441e-6ad2-4457-b1e0-e289e186f5dc None admin] PATH_INFO: `/v3/groups/8e985ee66ebd4ebfb87262d34bfec41f` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:22.084935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe441e-6ad2-4457-b1e0-e289e186f5dc None admin] RBAC: Authorizing `identity:get_group(group_id=8e985ee66ebd4ebfb87262d34bfec41f)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:22.086436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe441e-6ad2-4457-b1e0-e289e186f5dc None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:22.090443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 202/413] 10.222.0.88 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:21 2022] GET /identity/v3/groups/8e985ee66ebd4ebfb87262d34bfec41f => generated 229 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:22.091325 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe441e-6ad2-4457-b1e0-e289e186f5dc None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:22.134429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:22.181838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:22.202078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12f662-5c59-4bb6-b36d-0cc7054bc815 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:22.216151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe441e-6ad2-4457-b1e0-e289e186f5dc None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:22.254698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73e37e-1e2b-4af7-b437-5bcdfd113176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:22.255478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73e37e-1e2b-4af7-b437-5bcdfd113176 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:22.255763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73e37e-1e2b-4af7-b437-5bcdfd113176 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:22.256041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73e37e-1e2b-4af7-b437-5bcdfd113176 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:22.257284 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73e37e-1e2b-4af7-b437-5bcdfd113176 None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:22.258993 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73e37e-1e2b-4af7-b437-5bcdfd113176 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:22.259608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c26e7-f78b-4b85-b0d3-d979b7acf863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:22.260469 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c26e7-f78b-4b85-b0d3-d979b7acf863 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:22.260959 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c26e7-f78b-4b85-b0d3-d979b7acf863 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:22.261203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 212/414] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:22 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:22.261673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c26e7-f78b-4b85-b0d3-d979b7acf863 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:22.262031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73e37e-1e2b-4af7-b437-5bcdfd113176 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:22.275477 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c26e7-f78b-4b85-b0d3-d979b7acf863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:22.282883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-a72c26e7-f78b-4b85-b0d3-d979b7acf863 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:22.282883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:22.282883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:22.282883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:22.282883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:22.282883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:22.282883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:22.282883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:22.282883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:22.282883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:22.282883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:22.282883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:22.282883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:22.282883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:22.282883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:22.282883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:22.284056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:22.284056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:22.284056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:22.284056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:22.284056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:22.284056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:22.284056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:22.284056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:22.284056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:22.284056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:22.284056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:22.284056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:22.284056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:22.284056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:22.284056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:22.284056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:22.284984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:22.284984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:22.284984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:22.284984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:22.284984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:22.284984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:22.284984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:22.284984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:22.284984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:22.284984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:22.284984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:22.284984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:22.287789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 203/415] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:09:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:22.293802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c26e7-f78b-4b85-b0d3-d979b7acf863 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:22.385720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73e37e-1e2b-4af7-b437-5bcdfd113176 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:22.413131 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:22.420934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c26e7-f78b-4b85-b0d3-d979b7acf863 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:22.428151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bbc14-265f-4a98-bb11-67815f676ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:22.429090 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bbc14-265f-4a98-bb11-67815f676ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:22.429537 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bbc14-265f-4a98-bb11-67815f676ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:22.429961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bbc14-265f-4a98-bb11-67815f676ce0 None admin] PATH_INFO: `/v3/projects/fe428a23adc74bbab88ff935243cc4e2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:22.431514 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bbc14-265f-4a98-bb11-67815f676ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=fe428a23adc74bbab88ff935243cc4e2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:22.433356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bbc14-265f-4a98-bb11-67815f676ce0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:22.435812 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 213/416] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:22 2022] GET /identity/v3/projects/fe428a23adc74bbab88ff935243cc4e2 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:22.436917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bbc14-265f-4a98-bb11-67815f676ce0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:22.463872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba76a20-0100-4d5f-952b-4167df0731a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:22.464777 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba76a20-0100-4d5f-952b-4167df0731a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:22.465193 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba76a20-0100-4d5f-952b-4167df0731a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:22.466152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba76a20-0100-4d5f-952b-4167df0731a6 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a/groups/c6f84987009b42c4b25b0f33152fa58a/roles/3c32f171f34345e4958d31e3940092d6` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:22.469882 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba76a20-0100-4d5f-952b-4167df0731a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc2c956c8a3f4d0f832f927cdc74095a, group_id=c6f84987009b42c4b25b0f33152fa58a, role_id=3c32f171f34345e4958d31e3940092d6)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:22.471608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:22.473976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba76a20-0100-4d5f-952b-4167df0731a6 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:22.499018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 204/417] 10.222.0.88 () {58 vars in 1442 bytes} [Tue Aug 23 22:09:22 2022] PUT /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a/groups/c6f84987009b42c4b25b0f33152fa58a/roles/3c32f171f34345e4958d31e3940092d6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:22.500229 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba76a20-0100-4d5f-952b-4167df0731a6 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:22.618656 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bbc14-265f-4a98-bb11-67815f676ce0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:22.662563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66389c21-603e-4acd-bd83-c3e8560fc85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:22.663309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66389c21-603e-4acd-bd83-c3e8560fc85d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:22.663560 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66389c21-603e-4acd-bd83-c3e8560fc85d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:22.663901 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66389c21-603e-4acd-bd83-c3e8560fc85d None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:22.664920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66389c21-603e-4acd-bd83-c3e8560fc85d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:22.665276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66389c21-603e-4acd-bd83-c3e8560fc85d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:22.667120 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66389c21-603e-4acd-bd83-c3e8560fc85d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:22.670551 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba76a20-0100-4d5f-952b-4167df0731a6 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:22.676719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 214/418] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:22.682868 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66389c21-603e-4acd-bd83-c3e8560fc85d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:22.714257 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01458702-19fd-4d49-bbd5-7c49d39c96d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:22.715406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01458702-19fd-4d49-bbd5-7c49d39c96d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:22.715940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01458702-19fd-4d49-bbd5-7c49d39c96d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:22.716556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01458702-19fd-4d49-bbd5-7c49d39c96d9 None admin] PATH_INFO: `/v3/projects/fe428a23adc74bbab88ff935243cc4e2/groups/8e985ee66ebd4ebfb87262d34bfec41f/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:22.719175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01458702-19fd-4d49-bbd5-7c49d39c96d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe428a23adc74bbab88ff935243cc4e2, group_id=8e985ee66ebd4ebfb87262d34bfec41f, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:22.721416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01458702-19fd-4d49-bbd5-7c49d39c96d9 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:22.739059 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 205/419] 10.222.0.88 () {58 vars in 1442 bytes} [Tue Aug 23 22:09:22 2022] PUT /identity/v3/projects/fe428a23adc74bbab88ff935243cc4e2/groups/8e985ee66ebd4ebfb87262d34bfec41f/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:22.791381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:22.810900 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66389c21-603e-4acd-bd83-c3e8560fc85d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:22.856747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fdfe1b-9d82-4b3b-878d-b0e0564d6820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:22.857897 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdfe1b-9d82-4b3b-878d-b0e0564d6820 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:22.858380 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdfe1b-9d82-4b3b-878d-b0e0564d6820 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:22.858919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdfe1b-9d82-4b3b-878d-b0e0564d6820 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:22.986467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a002a02-ff86-4e56-ad95-6b87a28b832a None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:22.987625 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a002a02-ff86-4e56-ad95-6b87a28b832a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:22.988108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a002a02-ff86-4e56-ad95-6b87a28b832a None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:22.990360 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 206/420] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:22.996722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf55e85-a817-40f3-9547-9e29de435be6 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:22.997412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf55e85-a817-40f3-9547-9e29de435be6 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:22.997881 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf55e85-a817-40f3-9547-9e29de435be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:23.004528 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fdfe1b-9d82-4b3b-878d-b0e0564d6820 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:23.007772 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fdfe1b-9d82-4b3b-878d-b0e0564d6820 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:23.019599 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-ccf55e85-a817-40f3-9547-9e29de435be6 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:23.021345 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ccf55e85-a817-40f3-9547-9e29de435be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:23.035115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:23.134853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 207/421] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:23.217803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:23.264721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 215/422] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:23.273440 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf55e85-a817-40f3-9547-9e29de435be6 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:23.450889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf55e85-a817-40f3-9547-9e29de435be6 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:23.511513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc7e3f-2c43-47e4-a7e2-0f5352aec4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:23.512180 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc7e3f-2c43-47e4-a7e2-0f5352aec4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:23.512480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc7e3f-2c43-47e4-a7e2-0f5352aec4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:23.512752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc7e3f-2c43-47e4-a7e2-0f5352aec4b5 None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:23.513996 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc7e3f-2c43-47e4-a7e2-0f5352aec4b5 None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:23.515508 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc7e3f-2c43-47e4-a7e2-0f5352aec4b5 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:23.515853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fdfe1b-9d82-4b3b-878d-b0e0564d6820 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:23.517514 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 208/423] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:23 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:23.527703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc7e3f-2c43-47e4-a7e2-0f5352aec4b5 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:23.561433 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:23.695594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc7e3f-2c43-47e4-a7e2-0f5352aec4b5 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:23.788138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454017c-0446-468a-94af-a994d39f4962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:23.788738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454017c-0446-468a-94af-a994d39f4962 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:23.789021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454017c-0446-468a-94af-a994d39f4962 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:23.789325 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454017c-0446-468a-94af-a994d39f4962 None admin] PATH_INFO: `/v3/groups/c6f84987009b42c4b25b0f33152fa58a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:23.790755 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454017c-0446-468a-94af-a994d39f4962 None admin] RBAC: Authorizing `identity:get_group(group_id=c6f84987009b42c4b25b0f33152fa58a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:23.792202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454017c-0446-468a-94af-a994d39f4962 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:23.797176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 209/424] 10.222.0.88 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:23 2022] GET /identity/v3/groups/c6f84987009b42c4b25b0f33152fa58a => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:23.800940 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fdfe1b-9d82-4b3b-878d-b0e0564d6820 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:23.803826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454017c-0446-468a-94af-a994d39f4962 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:23.933817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9709b-581e-457f-8a94-3c0f1b5f2b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:23.934516 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9709b-581e-457f-8a94-3c0f1b5f2b38 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:23.934798 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9709b-581e-457f-8a94-3c0f1b5f2b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:23.935069 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9709b-581e-457f-8a94-3c0f1b5f2b38 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:23.969754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9709b-581e-457f-8a94-3c0f1b5f2b38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:23.969951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:23.975411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-e9b9709b-581e-457f-8a94-3c0f1b5f2b38 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:23.975411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:23.975411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:23.975411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:23.975411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:23.975411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:23.975411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:23.975411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:23.975411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:23.975411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:23.975411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:23.975411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:23.975411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:23.975411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:23.975411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:23.975411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:23.976344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:23.976344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:23.976344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:23.976344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:23.976344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:23.976344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:23.976344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:23.976344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:23.976344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:23.976344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:23.976344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:23.976344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:23.976344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:23.976344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:23.976344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:23.976344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:23.977159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:23.977159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:23.977159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:23.977159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:23.977159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:23.977159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:23.977159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:23.977159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:23.977159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:23.977159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:23.977159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:23.977159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:23.977159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 216/425] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 462 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:23.986252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9709b-581e-457f-8a94-3c0f1b5f2b38 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:24.000531 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454017c-0446-468a-94af-a994d39f4962 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:24.074357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c078cfa-3178-43cf-824f-a6c0b67c6032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:24.075280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c078cfa-3178-43cf-824f-a6c0b67c6032 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:24.075774 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c078cfa-3178-43cf-824f-a6c0b67c6032 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:24.076235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c078cfa-3178-43cf-824f-a6c0b67c6032 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:24.077653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c078cfa-3178-43cf-824f-a6c0b67c6032 None admin] RBAC: Adding query filter params (group.id=c6f84987009b42c4b25b0f33152fa58a, role.id=3c32f171f34345e4958d31e3940092d6, scope.project.id=709dbd6287a240f3b6044bf6d6e63334) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:24.078198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c078cfa-3178-43cf-824f-a6c0b67c6032 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:24.082613 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c078cfa-3178-43cf-824f-a6c0b67c6032 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:24.097394 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 210/426] 10.222.0.88 () {56 vars in 1376 bytes} [Tue Aug 23 22:09:23 2022] GET /identity/v3/role_assignments?group.id=c6f84987009b42c4b25b0f33152fa58a&scope.project.id=709dbd6287a240f3b6044bf6d6e63334&role.id=3c32f171f34345e4958d31e3940092d6 => generated 611 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:24.100506 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba946b-7ca6-43fe-ad1a-596c86cf55b4 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:24.101094 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba946b-7ca6-43fe-ad1a-596c86cf55b4 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:24.101379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba946b-7ca6-43fe-ad1a-596c86cf55b4 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:24.103217 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 211/427] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:24.111355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b135ee8-093f-41e0-91a4-2fbb2d1bc264 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:24.112369 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b135ee8-093f-41e0-91a4-2fbb2d1bc264 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:24.112834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b135ee8-093f-41e0-91a4-2fbb2d1bc264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:24.121468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:24.190988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9709b-581e-457f-8a94-3c0f1b5f2b38 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:24.198962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-7b135ee8-093f-41e0-91a4-2fbb2d1bc264 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:24.201106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-7b135ee8-093f-41e0-91a4-2fbb2d1bc264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:24.238621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a29499-66d3-42b7-acc6-0fa4eda9eb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:24.239281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a29499-66d3-42b7-acc6-0fa4eda9eb96 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:24.239532 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a29499-66d3-42b7-acc6-0fa4eda9eb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:24.239813 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a29499-66d3-42b7-acc6-0fa4eda9eb96 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:24.240533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a29499-66d3-42b7-acc6-0fa4eda9eb96 None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:24.240842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a29499-66d3-42b7-acc6-0fa4eda9eb96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:24.243147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a29499-66d3-42b7-acc6-0fa4eda9eb96 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:24.254782 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 217/428] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:24.267392 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a29499-66d3-42b7-acc6-0fa4eda9eb96 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:24.380101 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 212/429] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:24.453245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a29499-66d3-42b7-acc6-0fa4eda9eb96 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:24.469272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:24.505257 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5071f6-7951-44d5-b9dd-cf3e977b85c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:24.506253 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5071f6-7951-44d5-b9dd-cf3e977b85c2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:24.506731 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5071f6-7951-44d5-b9dd-cf3e977b85c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:24.507229 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5071f6-7951-44d5-b9dd-cf3e977b85c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:24.605644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5071f6-7951-44d5-b9dd-cf3e977b85c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:24.608285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5071f6-7951-44d5-b9dd-cf3e977b85c2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:24.754667 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 218/430] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 22:09:24.761159 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b135ee8-093f-41e0-91a4-2fbb2d1bc264 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:24.871635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:24.894681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5071f6-7951-44d5-b9dd-cf3e977b85c2 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:24.927606 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b135ee8-093f-41e0-91a4-2fbb2d1bc264 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:24.943979 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:24.979958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6cbf4-c993-440b-b0a5-e2ea9a755ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:24.980671 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6cbf4-c993-440b-b0a5-e2ea9a755ecf None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:24.981017 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6cbf4-c993-440b-b0a5-e2ea9a755ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:24.981337 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6cbf4-c993-440b-b0a5-e2ea9a755ecf None admin] PATH_INFO: `/v3/projects/709dbd6287a240f3b6044bf6d6e63334` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:24.982760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6cbf4-c993-440b-b0a5-e2ea9a755ecf None admin] RBAC: Authorizing `identity:get_project(project_id=709dbd6287a240f3b6044bf6d6e63334)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:24.984628 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6cbf4-c993-440b-b0a5-e2ea9a755ecf None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:24.987348 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 213/431] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:24 2022] GET /identity/v3/projects/709dbd6287a240f3b6044bf6d6e63334 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:24.996305 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6cbf4-c993-440b-b0a5-e2ea9a755ecf None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:25.093442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5071f6-7951-44d5-b9dd-cf3e977b85c2 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:25.163012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a8aec-4550-442b-b49f-dc2ffd3f2e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:25.164151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a8aec-4550-442b-b49f-dc2ffd3f2e53 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:25.164645 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a8aec-4550-442b-b49f-dc2ffd3f2e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:25.165161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a8aec-4550-442b-b49f-dc2ffd3f2e53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:25.186798 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a8aec-4550-442b-b49f-dc2ffd3f2e53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:25.197938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6cbf4-c993-440b-b0a5-e2ea9a755ecf None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:25.198190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-8a2a8aec-4550-442b-b49f-dc2ffd3f2e53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:25.198190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:25.198190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:25.198190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:25.198190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:25.198190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:25.198190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:25.198190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:25.198190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:25.198190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:25.198190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:25.198190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:25.198190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:25.198190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:25.198190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:25.198190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:25.199804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:25.199804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:25.199804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:25.199804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:25.199804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:25.199804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:25.199804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:25.199804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:25.199804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:25.199804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:25.199804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:25.199804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:25.199804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:25.199804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:25.199804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:25.199804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:25.202276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:25.202276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:25.202276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:25.202276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:25.202276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:25.202276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:25.202276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:25.202276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:25.202276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:25.202276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:25.202276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:25.202276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:25.202276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 219/432] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:25.214723 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a8aec-4550-442b-b49f-dc2ffd3f2e53 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:25.265323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac97af5-49ee-4cf3-a131-ccdb77ed99f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:25.266510 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac97af5-49ee-4cf3-a131-ccdb77ed99f1 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:25.267006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac97af5-49ee-4cf3-a131-ccdb77ed99f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:25.267557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac97af5-49ee-4cf3-a131-ccdb77ed99f1 None admin] PATH_INFO: `/v3/groups/c6f84987009b42c4b25b0f33152fa58a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:25.270175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac97af5-49ee-4cf3-a131-ccdb77ed99f1 None admin] RBAC: Authorizing `identity:get_group(group_id=c6f84987009b42c4b25b0f33152fa58a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:25.273581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac97af5-49ee-4cf3-a131-ccdb77ed99f1 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:25.281579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 214/433] 10.222.0.88 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:24 2022] GET /identity/v3/groups/c6f84987009b42c4b25b0f33152fa58a => generated 226 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:25.286976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac97af5-49ee-4cf3-a131-ccdb77ed99f1 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:25.329687 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:25.352547 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:25.424965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a8aec-4550-442b-b49f-dc2ffd3f2e53 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:25.493193 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac97af5-49ee-4cf3-a131-ccdb77ed99f1 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:25.493645 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44692c-a29c-4436-85e6-99ddd156108e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:25.494687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44692c-a29c-4436-85e6-99ddd156108e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:25.495185 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44692c-a29c-4436-85e6-99ddd156108e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:25.495680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44692c-a29c-4436-85e6-99ddd156108e None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:25.496957 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44692c-a29c-4436-85e6-99ddd156108e None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:25.497681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44692c-a29c-4436-85e6-99ddd156108e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:25.500480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44692c-a29c-4436-85e6-99ddd156108e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:25.519693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 220/434] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:25.529232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44692c-a29c-4436-85e6-99ddd156108e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:25.569808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f695b-3581-4cdc-b48a-168a50076914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:25.571176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f695b-3581-4cdc-b48a-168a50076914 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:25.571847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f695b-3581-4cdc-b48a-168a50076914 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:25.572485 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f695b-3581-4cdc-b48a-168a50076914 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:25.574444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f695b-3581-4cdc-b48a-168a50076914 None admin] RBAC: Adding query filter params (group.id=c6f84987009b42c4b25b0f33152fa58a, role.id=fad7ba55962b40459b40dcc2bad395b2, scope.project.id=709dbd6287a240f3b6044bf6d6e63334) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:25.575397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f695b-3581-4cdc-b48a-168a50076914 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:25.578609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f695b-3581-4cdc-b48a-168a50076914 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:25.597851 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 215/435] 10.222.0.88 () {56 vars in 1376 bytes} [Tue Aug 23 22:09:25 2022] GET /identity/v3/role_assignments?group.id=c6f84987009b42c4b25b0f33152fa58a&scope.project.id=709dbd6287a240f3b6044bf6d6e63334&role.id=fad7ba55962b40459b40dcc2bad395b2 => generated 611 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:25.757818 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44692c-a29c-4436-85e6-99ddd156108e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:25.758079 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3509df-4cf4-40f7-9a0d-609ba9127e42 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:25.758407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3509df-4cf4-40f7-9a0d-609ba9127e42 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:25.758693 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3509df-4cf4-40f7-9a0d-609ba9127e42 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:25.760392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 216/436] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:09:25.768636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadc37f-9762-4a7c-9aa9-090c64fa2bff None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:25.769299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadc37f-9762-4a7c-9aa9-090c64fa2bff None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:25.769621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadc37f-9762-4a7c-9aa9-090c64fa2bff None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:25.773440 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 217/437] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:25.780328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef26273-469c-448e-b231-2a15b478a9aa None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:25.780328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef26273-469c-448e-b231-2a15b478a9aa None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:25.780328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef26273-469c-448e-b231-2a15b478a9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:25.801098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:25.805281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43213468-189d-4a9d-afae-e3a4d5b1d9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:25.805914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43213468-189d-4a9d-afae-e3a4d5b1d9bf None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:25.806372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43213468-189d-4a9d-afae-e3a4d5b1d9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:25.806514 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43213468-189d-4a9d-afae-e3a4d5b1d9bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:25.818809 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-eef26273-469c-448e-b231-2a15b478a9aa None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:25.821773 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-eef26273-469c-448e-b231-2a15b478a9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:25.932925 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43213468-189d-4a9d-afae-e3a4d5b1d9bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:25.935938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43213468-189d-4a9d-afae-e3a4d5b1d9bf None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:25.948586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 218/438] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:25.951312 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec6dba-0720-43eb-bdb0-1cb5cd77c322 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:25.952199 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec6dba-0720-43eb-bdb0-1cb5cd77c322 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:25.953723 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec6dba-0720-43eb-bdb0-1cb5cd77c322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:25.976767 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:25.985858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-a5ec6dba-0720-43eb-bdb0-1cb5cd77c322 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:25.988006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a5ec6dba-0720-43eb-bdb0-1cb5cd77c322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:26.129195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 219/439] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:26.152805 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 221/440] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:26.160384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec6dba-0720-43eb-bdb0-1cb5cd77c322 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:26.318726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43213468-189d-4a9d-afae-e3a4d5b1d9bf None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:26.369155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec6dba-0720-43eb-bdb0-1cb5cd77c322 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:26.384763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:26.443500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f63f1-f52a-4c53-9bc6-270369eed136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:26.444894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f63f1-f52a-4c53-9bc6-270369eed136 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:26.445568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f63f1-f52a-4c53-9bc6-270369eed136 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:26.446123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f63f1-f52a-4c53-9bc6-270369eed136 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:26.450309 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f63f1-f52a-4c53-9bc6-270369eed136 None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:26.452373 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f63f1-f52a-4c53-9bc6-270369eed136 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:26.457193 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 220/441] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:26 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:26.464159 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f63f1-f52a-4c53-9bc6-270369eed136 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:26.475090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:26.553815 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43213468-189d-4a9d-afae-e3a4d5b1d9bf None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:26.605361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417aa87-ce72-4026-9a8f-556147826bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:26.606165 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417aa87-ce72-4026-9a8f-556147826bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:26.606642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417aa87-ce72-4026-9a8f-556147826bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:26.606994 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417aa87-ce72-4026-9a8f-556147826bd1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:26.616810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f63f1-f52a-4c53-9bc6-270369eed136 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:26.622028 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417aa87-ce72-4026-9a8f-556147826bd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:26.627982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-5417aa87-ce72-4026-9a8f-556147826bd1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:26.627982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:26.627982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:26.627982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:26.627982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:26.627982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:26.627982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:26.627982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:26.627982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:26.627982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:26.627982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:26.627982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:26.627982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:26.627982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:26.627982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:26.627982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:26.629241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:26.629241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:26.629241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:26.629241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:26.629241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:26.629241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:26.629241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:26.629241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:26.629241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:26.629241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:26.629241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:26.629241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:26.629241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:26.629241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:26.629241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:26.629241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:26.630762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:26.630762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:26.630762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:26.630762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:26.630762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:26.630762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:26.630762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:26.630762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:26.630762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:26.630762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:26.630762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:26.630762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:26.630762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 222/442] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:09:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:26.630762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417aa87-ce72-4026-9a8f-556147826bd1 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:26.664802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63a826-9042-4f84-a515-da60a01e6732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:26.665616 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63a826-9042-4f84-a515-da60a01e6732 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:26.666016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63a826-9042-4f84-a515-da60a01e6732 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:26.666377 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63a826-9042-4f84-a515-da60a01e6732 None admin] PATH_INFO: `/v3/groups/c6f84987009b42c4b25b0f33152fa58a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:26.667758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63a826-9042-4f84-a515-da60a01e6732 None admin] RBAC: Authorizing `identity:get_group(group_id=c6f84987009b42c4b25b0f33152fa58a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:26.669535 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63a826-9042-4f84-a515-da60a01e6732 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:26.671579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 221/443] 10.222.0.88 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:26 2022] GET /identity/v3/groups/c6f84987009b42c4b25b0f33152fa58a => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:26.672466 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63a826-9042-4f84-a515-da60a01e6732 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:26.782181 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:26.800251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417aa87-ce72-4026-9a8f-556147826bd1 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:26.806271 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:26.844725 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb40e69-ce3c-469a-896e-edb17032b26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:26.845548 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb40e69-ce3c-469a-896e-edb17032b26a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:26.846677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb40e69-ce3c-469a-896e-edb17032b26a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:26.847689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb40e69-ce3c-469a-896e-edb17032b26a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:26.861746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb40e69-ce3c-469a-896e-edb17032b26a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:26.867856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-cfb40e69-ce3c-469a-896e-edb17032b26a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:26.867856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:26.867856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:26.867856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:26.867856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:26.867856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:26.867856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:26.867856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:26.867856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:26.867856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:26.867856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:26.867856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:26.867856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:26.867856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:26.867856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:26.867856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:26.869709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:26.869709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:26.869709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:26.869709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:26.869709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:26.869709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:26.869709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:26.869709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:26.869709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:26.869709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:26.869709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:26.869709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:26.869709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:26.869709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:26.869709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:26.869709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:26.871324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:26.871324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:26.871324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:26.871324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:26.871324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:26.871324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:26.871324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:26.871324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:26.871324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:26.871324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:26.871324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:26.871324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:26.873422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63a826-9042-4f84-a515-da60a01e6732 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:26.873610 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 223/444] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:09:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:26.873610 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb40e69-ce3c-469a-896e-edb17032b26a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:26.927776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3845d0-7a6e-4b22-b238-e82e53924bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:26.928468 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3845d0-7a6e-4b22-b238-e82e53924bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:26.928837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3845d0-7a6e-4b22-b238-e82e53924bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:26.929271 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3845d0-7a6e-4b22-b238-e82e53924bd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:26.930061 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3845d0-7a6e-4b22-b238-e82e53924bd0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:26.930478 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3845d0-7a6e-4b22-b238-e82e53924bd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:26.932065 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3845d0-7a6e-4b22-b238-e82e53924bd0 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:26.943599 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 222/445] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:26.943599 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3845d0-7a6e-4b22-b238-e82e53924bd0 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:27.058693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb40e69-ce3c-469a-896e-edb17032b26a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:27.078788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3845d0-7a6e-4b22-b238-e82e53924bd0 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:27.114864 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa57ff-8d6f-46aa-90f1-b451637dc656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:27.115907 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa57ff-8d6f-46aa-90f1-b451637dc656 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:27.116399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa57ff-8d6f-46aa-90f1-b451637dc656 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:27.116887 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa57ff-8d6f-46aa-90f1-b451637dc656 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:27.118303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa57ff-8d6f-46aa-90f1-b451637dc656 None admin] RBAC: Adding query filter params (group.id=c6f84987009b42c4b25b0f33152fa58a, role.id=fad7ba55962b40459b40dcc2bad395b2, scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:27.118866 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa57ff-8d6f-46aa-90f1-b451637dc656 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:27.120086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dae6d-465e-43f3-87d6-6c29616369b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:27.120709 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dae6d-465e-43f3-87d6-6c29616369b4 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:27.120986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dae6d-465e-43f3-87d6-6c29616369b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:27.121290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dae6d-465e-43f3-87d6-6c29616369b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:27.121684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa57ff-8d6f-46aa-90f1-b451637dc656 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:27.121970 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028dae6d-465e-43f3-87d6-6c29616369b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:27.122376 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028dae6d-465e-43f3-87d6-6c29616369b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:27.123663 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028dae6d-465e-43f3-87d6-6c29616369b4 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:27.125432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:27.133141 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 224/446] 10.222.0.88 () {56 vars in 1376 bytes} [Tue Aug 23 22:09:26 2022] GET /identity/v3/role_assignments?group.id=c6f84987009b42c4b25b0f33152fa58a&scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a&role.id=fad7ba55962b40459b40dcc2bad395b2 => generated 611 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:27.134742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa57ff-8d6f-46aa-90f1-b451637dc656 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:27.139632 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 223/447] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:27.141461 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dae6d-465e-43f3-87d6-6c29616369b4 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:27.166910 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:27.309478 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dae6d-465e-43f3-87d6-6c29616369b4 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:27.343487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa57ff-8d6f-46aa-90f1-b451637dc656 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:27.374154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e39fca-737b-46fe-8b92-5f585d37ce50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:27.375125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e39fca-737b-46fe-8b92-5f585d37ce50 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:27.375570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e39fca-737b-46fe-8b92-5f585d37ce50 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:27.375983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e39fca-737b-46fe-8b92-5f585d37ce50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:27.411081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d5a84-a7d5-4dba-aec7-2e76a50f7fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:27.411910 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d5a84-a7d5-4dba-aec7-2e76a50f7fba None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:27.412384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d5a84-a7d5-4dba-aec7-2e76a50f7fba None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:27.412879 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d5a84-a7d5-4dba-aec7-2e76a50f7fba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:27.501222 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:27.515557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e39fca-737b-46fe-8b92-5f585d37ce50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:27.518685 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e39fca-737b-46fe-8b92-5f585d37ce50 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:27.549755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d5a84-a7d5-4dba-aec7-2e76a50f7fba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:27.552089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d5a84-a7d5-4dba-aec7-2e76a50f7fba None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:27.577851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:27.696883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 225/448] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:27.711948 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 224/449] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 22:09:27.712247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d5a84-a7d5-4dba-aec7-2e76a50f7fba None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:27.713628 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e39fca-737b-46fe-8b92-5f585d37ce50 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:27.872847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d5a84-a7d5-4dba-aec7-2e76a50f7fba None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:27.876338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:27.886538 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e39fca-737b-46fe-8b92-5f585d37ce50 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:27.893563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:27.929956 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee33d1-e76b-4d32-b76c-3fe86a6b9c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:27.931050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee33d1-e76b-4d32-b76c-3fe86a6b9c89 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:27.931530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee33d1-e76b-4d32-b76c-3fe86a6b9c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:27.932008 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee33d1-e76b-4d32-b76c-3fe86a6b9c89 None admin] PATH_INFO: `/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:27.934046 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee33d1-e76b-4d32-b76c-3fe86a6b9c89 None admin] RBAC: Authorizing `identity:get_project(project_id=dc2c956c8a3f4d0f832f927cdc74095a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:27.936779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee33d1-e76b-4d32-b76c-3fe86a6b9c89 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:27.945531 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 226/450] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:27 2022] GET /identity/v3/projects/dc2c956c8a3f4d0f832f927cdc74095a => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:27.947341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee33d1-e76b-4d32-b76c-3fe86a6b9c89 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:27.954528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc718e-8c64-4fbd-9380-ffa8ea101573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:27.957481 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc718e-8c64-4fbd-9380-ffa8ea101573 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:27.957981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc718e-8c64-4fbd-9380-ffa8ea101573 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:27.958526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc718e-8c64-4fbd-9380-ffa8ea101573 None admin] PATH_INFO: `/v3/projects/fe428a23adc74bbab88ff935243cc4e2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:27.960751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc718e-8c64-4fbd-9380-ffa8ea101573 None admin] RBAC: Authorizing `identity:get_project(project_id=fe428a23adc74bbab88ff935243cc4e2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:27.963636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc718e-8c64-4fbd-9380-ffa8ea101573 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:27.967139 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 225/451] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:27 2022] GET /identity/v3/projects/fe428a23adc74bbab88ff935243cc4e2 => generated 344 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:27.976254 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc718e-8c64-4fbd-9380-ffa8ea101573 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:28.091734 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc718e-8c64-4fbd-9380-ffa8ea101573 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:28.123981 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee33d1-e76b-4d32-b76c-3fe86a6b9c89 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:28.132024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ee555-8a36-43a7-8fba-740bf3658ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:28.132790 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ee555-8a36-43a7-8fba-740bf3658ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:28.133262 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ee555-8a36-43a7-8fba-740bf3658ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:28.133642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ee555-8a36-43a7-8fba-740bf3658ca4 None admin] PATH_INFO: `/v3/groups/8e985ee66ebd4ebfb87262d34bfec41f` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:28.135020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ee555-8a36-43a7-8fba-740bf3658ca4 None admin] RBAC: Authorizing `identity:get_group(group_id=8e985ee66ebd4ebfb87262d34bfec41f)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:28.136844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ee555-8a36-43a7-8fba-740bf3658ca4 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:28.138732 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 226/452] 10.222.0.88 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:27 2022] GET /identity/v3/groups/8e985ee66ebd4ebfb87262d34bfec41f => generated 229 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:28.145883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ee555-8a36-43a7-8fba-740bf3658ca4 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:28.168014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795abd3-bacd-477c-9b94-a1f770b3d7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:28.168650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795abd3-bacd-477c-9b94-a1f770b3d7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:28.168934 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795abd3-bacd-477c-9b94-a1f770b3d7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:28.169244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795abd3-bacd-477c-9b94-a1f770b3d7c6 None admin] PATH_INFO: `/v3/groups/c6f84987009b42c4b25b0f33152fa58a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:28.170444 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795abd3-bacd-477c-9b94-a1f770b3d7c6 None admin] RBAC: Authorizing `identity:get_group(group_id=c6f84987009b42c4b25b0f33152fa58a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:28.171943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795abd3-bacd-477c-9b94-a1f770b3d7c6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:28.173636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 227/453] 10.222.0.88 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:27 2022] GET /identity/v3/groups/c6f84987009b42c4b25b0f33152fa58a => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:28.184042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795abd3-bacd-477c-9b94-a1f770b3d7c6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:28.190781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:28.277582 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:28.282188 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ee555-8a36-43a7-8fba-740bf3658ca4 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:28.338312 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49adfb48-b616-46a0-9270-83953935f5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:28.339253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49adfb48-b616-46a0-9270-83953935f5f3 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:28.339699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49adfb48-b616-46a0-9270-83953935f5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:28.340134 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49adfb48-b616-46a0-9270-83953935f5f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:28.341434 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49adfb48-b616-46a0-9270-83953935f5f3 None admin] RBAC: Adding query filter params (group.id=8e985ee66ebd4ebfb87262d34bfec41f, role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=fe428a23adc74bbab88ff935243cc4e2) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:28.341943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49adfb48-b616-46a0-9270-83953935f5f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:28.344377 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49adfb48-b616-46a0-9270-83953935f5f3 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:28.354766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 227/454] 10.222.0.88 () {56 vars in 1376 bytes} [Tue Aug 23 22:09:28 2022] GET /identity/v3/role_assignments?group.id=8e985ee66ebd4ebfb87262d34bfec41f&scope.project.id=fe428a23adc74bbab88ff935243cc4e2&role.id=dc5d71ab685e420697f520aa3a368177 => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:28.356383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795abd3-bacd-477c-9b94-a1f770b3d7c6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:28.415779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37506068-6e2f-4f94-949f-4ad69e56bbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:28.416825 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37506068-6e2f-4f94-949f-4ad69e56bbc6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:28.417224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37506068-6e2f-4f94-949f-4ad69e56bbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:28.417523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37506068-6e2f-4f94-949f-4ad69e56bbc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:28.418336 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37506068-6e2f-4f94-949f-4ad69e56bbc6 None admin] RBAC: Adding query filter params (group.id=c6f84987009b42c4b25b0f33152fa58a, role.id=3c32f171f34345e4958d31e3940092d6, scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:28.418646 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37506068-6e2f-4f94-949f-4ad69e56bbc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:28.420720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37506068-6e2f-4f94-949f-4ad69e56bbc6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:28.430365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 228/455] 10.222.0.88 () {56 vars in 1376 bytes} [Tue Aug 23 22:09:28 2022] GET /identity/v3/role_assignments?group.id=c6f84987009b42c4b25b0f33152fa58a&scope.project.id=dc2c956c8a3f4d0f832f927cdc74095a&role.id=3c32f171f34345e4958d31e3940092d6 => generated 611 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:31.833365 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba13ba4-63d5-49bf-a94a-f416b19a3e32 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:31.834233 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba13ba4-63d5-49bf-a94a-f416b19a3e32 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:31.834233 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba13ba4-63d5-49bf-a94a-f416b19a3e32 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:31.837093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 228/456] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:31.843547 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd507329-e8bb-422f-9d39-179df471f1b3 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:31.844299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd507329-e8bb-422f-9d39-179df471f1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:31.844426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd507329-e8bb-422f-9d39-179df471f1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:31.885361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fd507329-e8bb-422f-9d39-179df471f1b3 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:31.887358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fd507329-e8bb-422f-9d39-179df471f1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:32.050829 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 229/457] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:32.286833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4fde0-8d65-4f05-8a70-d2351914a63c None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:32.287510 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4fde0-8d65-4f05-8a70-d2351914a63c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:32.287995 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4fde0-8d65-4f05-8a70-d2351914a63c None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:32.289985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 229/458] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:32.300009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f81963-687c-472e-a25f-9fb7be432548 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:32.300527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f81963-687c-472e-a25f-9fb7be432548 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:32.301112 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f81963-687c-472e-a25f-9fb7be432548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:32.340987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-57f81963-687c-472e-a25f-9fb7be432548 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:32.343100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-57f81963-687c-472e-a25f-9fb7be432548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:32.449090 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:32.463307 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4fde0-8d65-4f05-8a70-d2351914a63c None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:32.523875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 230/459] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:32.534445 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f22b9-9a8e-4d60-b63d-764a6e58013e None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:32.535167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f22b9-9a8e-4d60-b63d-764a6e58013e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:32.535665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f22b9-9a8e-4d60-b63d-764a6e58013e None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:32.538906 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 231/460] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:32 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:32.549264 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f33e22-46ed-4c4b-ae6c-6f11b00ed8bd None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:32.549264 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f33e22-46ed-4c4b-ae6c-6f11b00ed8bd None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:32.549264 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f33e22-46ed-4c4b-ae6c-6f11b00ed8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:32.592922 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-45f33e22-46ed-4c4b-ae6c-6f11b00ed8bd None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:32.594074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-45f33e22-46ed-4c4b-ae6c-6f11b00ed8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:32.623670 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:32.637566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4fde0-8d65-4f05-8a70-d2351914a63c None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:32.705329 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf7bd9-2107-4a36-86ae-8ecc00624b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:32.706380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf7bd9-2107-4a36-86ae-8ecc00624b8e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:32.706848 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf7bd9-2107-4a36-86ae-8ecc00624b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:32.707315 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf7bd9-2107-4a36-86ae-8ecc00624b8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:32.728068 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf7bd9-2107-4a36-86ae-8ecc00624b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:32.730927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf7bd9-2107-4a36-86ae-8ecc00624b8e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:32.760205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-20cf7bd9-2107-4a36-86ae-8ecc00624b8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:32.760205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:32.760205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:32.760205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:32.760205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:32.760205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:32.760205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:32.760205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:32.760205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:32.760205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:32.760205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:32.760205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:32.760205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:32.760205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:32.760205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:32.760205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:32.761249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:32.762106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:32.762106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:32.762106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:32.762756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 230/461] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:09:32.770443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf7bd9-2107-4a36-86ae-8ecc00624b8e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:32.776572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 232/462] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:32.790691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45634ed-ec15-4304-b698-18a0f1b5ae80 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:32.791224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45634ed-ec15-4304-b698-18a0f1b5ae80 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:32.791716 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45634ed-ec15-4304-b698-18a0f1b5ae80 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:32.795033 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 233/463] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:32.805284 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920d060-1b20-4a84-80a9-f0e8a18330e1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:32.806012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920d060-1b20-4a84-80a9-f0e8a18330e1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:32.806636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920d060-1b20-4a84-80a9-f0e8a18330e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:32.861606 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-1920d060-1b20-4a84-80a9-f0e8a18330e1 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:32.863663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1920d060-1b20-4a84-80a9-f0e8a18330e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:33.077426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 234/464] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:33.099176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf7bd9-2107-4a36-86ae-8ecc00624b8e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:33.116441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920d060-1b20-4a84-80a9-f0e8a18330e1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:33.197978 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9b8bb-01e9-44ef-8d75-a1f7c014be74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:33.198951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9b8bb-01e9-44ef-8d75-a1f7c014be74 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:33.199428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9b8bb-01e9-44ef-8d75-a1f7c014be74 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:33.199925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9b8bb-01e9-44ef-8d75-a1f7c014be74 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:33.201190 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9b8bb-01e9-44ef-8d75-a1f7c014be74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:33.201741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9b8bb-01e9-44ef-8d75-a1f7c014be74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:33.202635 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:33.204366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9b8bb-01e9-44ef-8d75-a1f7c014be74 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:33.229312 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 231/465] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:33.230857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9b8bb-01e9-44ef-8d75-a1f7c014be74 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:33.251395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:33.343700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920d060-1b20-4a84-80a9-f0e8a18330e1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:33.407466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe219a-f4c3-43f0-bcce-7381055d9fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:33.409046 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe219a-f4c3-43f0-bcce-7381055d9fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:33.409561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe219a-f4c3-43f0-bcce-7381055d9fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:33.416441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe219a-f4c3-43f0-bcce-7381055d9fe5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:33.423977 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9b8bb-01e9-44ef-8d75-a1f7c014be74 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:33.444948 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe219a-f4c3-43f0-bcce-7381055d9fe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:33.448127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe219a-f4c3-43f0-bcce-7381055d9fe5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:33.481293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-60fe219a-f4c3-43f0-bcce-7381055d9fe5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:33.481293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:33.481293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:33.481293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:33.481293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:33.481293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:33.481293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:33.481293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:33.481293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:33.481293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:33.481293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:33.481293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:33.481293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:33.481293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:33.481293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:33.481293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:33.482239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:33.483101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:33.483101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:33.483101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:33.489311 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 235/466] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:33.490373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe219a-f4c3-43f0-bcce-7381055d9fe5 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:33.491044 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684db38a-1d94-411f-a710-2a2be764a392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:33.492156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684db38a-1d94-411f-a710-2a2be764a392 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:33.492733 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684db38a-1d94-411f-a710-2a2be764a392 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:33.493259 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684db38a-1d94-411f-a710-2a2be764a392 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:33.515502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684db38a-1d94-411f-a710-2a2be764a392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:33.518233 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684db38a-1d94-411f-a710-2a2be764a392 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:33.542035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-684db38a-1d94-411f-a710-2a2be764a392 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:33.542035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:33.542035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:33.542035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:33.542035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:33.542035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:33.542035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:33.542035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:33.542035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:33.542035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:33.542035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:33.542035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:33.542035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:33.542035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:33.542035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:33.542035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:33.543830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:33.545819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:33.545819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:33.545819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:33.545819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 232/467] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:33.545819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684db38a-1d94-411f-a710-2a2be764a392 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:33.601404 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:33.626167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe219a-f4c3-43f0-bcce-7381055d9fe5 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:33.628717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:33.677288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684db38a-1d94-411f-a710-2a2be764a392 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:33.687615 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc496d47-c8de-439c-b2fc-57562f3f4760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:33.688643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc496d47-c8de-439c-b2fc-57562f3f4760 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:33.690164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc496d47-c8de-439c-b2fc-57562f3f4760 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:33.691492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc496d47-c8de-439c-b2fc-57562f3f4760 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:33.692714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc496d47-c8de-439c-b2fc-57562f3f4760 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:33.695557 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc496d47-c8de-439c-b2fc-57562f3f4760 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:33.716909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c741a6-f510-4365-a970-35630e9bce16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:33.717218 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c741a6-f510-4365-a970-35630e9bce16 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:33.717563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c741a6-f510-4365-a970-35630e9bce16 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:33.717783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c741a6-f510-4365-a970-35630e9bce16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:33.730937 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 236/468] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:09:33 2022] POST /identity/v3/users => generated 248 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:09:33.732395 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc496d47-c8de-439c-b2fc-57562f3f4760 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:33.740429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c741a6-f510-4365-a970-35630e9bce16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:33.743001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c741a6-f510-4365-a970-35630e9bce16 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:33.770907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-94c741a6-f510-4365-a970-35630e9bce16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:33.770907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:33.770907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:33.770907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:33.770907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:33.770907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:33.770907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:33.770907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:33.770907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:33.770907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:33.770907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:33.770907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:33.770907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:33.770907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:33.770907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:33.770907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:33.771888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:33.772778 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:33.772778 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:33.772778 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:33.773721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 233/469] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:33.775233 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f76bb2-e802-4892-80d9-a895903cf8b3 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:33.775665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f76bb2-e802-4892-80d9-a895903cf8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:33.776088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f76bb2-e802-4892-80d9-a895903cf8b3 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:33.781110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 234/470] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:33.781110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f76bb2-e802-4892-80d9-a895903cf8b3 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:33.949082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc496d47-c8de-439c-b2fc-57562f3f4760 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:33.964304 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f76bb2-e802-4892-80d9-a895903cf8b3 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:33.988674 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:34.016085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d5e0a-52fc-4fc9-979d-6c8e04cce8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:34.017223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d5e0a-52fc-4fc9-979d-6c8e04cce8ed None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:34.018304 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d5e0a-52fc-4fc9-979d-6c8e04cce8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:34.018799 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d5e0a-52fc-4fc9-979d-6c8e04cce8ed None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:34.020016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d5e0a-52fc-4fc9-979d-6c8e04cce8ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:34.021193 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d5e0a-52fc-4fc9-979d-6c8e04cce8ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:34.023610 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d5e0a-52fc-4fc9-979d-6c8e04cce8ed None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:34.041096 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36379d95-f944-47cb-86ff-270c1e8ff3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:34.042064 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36379d95-f944-47cb-86ff-270c1e8ff3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:34.042484 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36379d95-f944-47cb-86ff-270c1e8ff3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:34.042929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36379d95-f944-47cb-86ff-270c1e8ff3b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:34.043746 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36379d95-f944-47cb-86ff-270c1e8ff3b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:34.044235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36379d95-f944-47cb-86ff-270c1e8ff3b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:34.046008 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36379d95-f944-47cb-86ff-270c1e8ff3b6 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:34.060381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 237/471] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:34.062134 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d5e0a-52fc-4fc9-979d-6c8e04cce8ed None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:34.068009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:34.072645 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 235/472] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:34.072645 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1070846-b68d-433d-be99-a54b8e3661ec None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:34.072645 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1070846-b68d-433d-be99-a54b8e3661ec None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:34.072645 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1070846-b68d-433d-be99-a54b8e3661ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:34.103396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-e1070846-b68d-433d-be99-a54b8e3661ec None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:34.105254 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e1070846-b68d-433d-be99-a54b8e3661ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:34.226252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 236/473] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:34.228243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1070846-b68d-433d-be99-a54b8e3661ec None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:34.285577 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d5e0a-52fc-4fc9-979d-6c8e04cce8ed None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:34.360677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a5f09-0687-46b1-b8c4-ae930f2624b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:34.362505 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a5f09-0687-46b1-b8c4-ae930f2624b2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:34.363208 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a5f09-0687-46b1-b8c4-ae930f2624b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:34.363992 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a5f09-0687-46b1-b8c4-ae930f2624b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:34.365461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a5f09-0687-46b1-b8c4-ae930f2624b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:34.366205 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a5f09-0687-46b1-b8c4-ae930f2624b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:34.369103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a5f09-0687-46b1-b8c4-ae930f2624b2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:34.395785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 238/474] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:34.397749 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a5f09-0687-46b1-b8c4-ae930f2624b2 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:34.428657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1070846-b68d-433d-be99-a54b8e3661ec None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:34.431892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:34.497277 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56434403-c02b-4667-8583-34d32d20258e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:34.498445 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56434403-c02b-4667-8583-34d32d20258e None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:34.499044 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56434403-c02b-4667-8583-34d32d20258e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:34.499636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56434403-c02b-4667-8583-34d32d20258e None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:34.500918 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56434403-c02b-4667-8583-34d32d20258e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:34.504469 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56434403-c02b-4667-8583-34d32d20258e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:34.532556 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:34.537782 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 237/475] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:09:34 2022] POST /identity/v3/users => generated 248 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:09:34.538488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56434403-c02b-4667-8583-34d32d20258e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:34.617549 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a5f09-0687-46b1-b8c4-ae930f2624b2 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:34.661804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143e5d9-93e4-48d7-8c08-e650f019d05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:34.662447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143e5d9-93e4-48d7-8c08-e650f019d05e None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:34.662908 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143e5d9-93e4-48d7-8c08-e650f019d05e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:34.663326 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143e5d9-93e4-48d7-8c08-e650f019d05e None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:34.664009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143e5d9-93e4-48d7-8c08-e650f019d05e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:34.665942 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143e5d9-93e4-48d7-8c08-e650f019d05e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:34.699565 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 239/476] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:09:34 2022] POST /identity/v3/users => generated 246 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:09:34.700506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143e5d9-93e4-48d7-8c08-e650f019d05e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:34.804144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56434403-c02b-4667-8583-34d32d20258e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:34.852816 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143e5d9-93e4-48d7-8c08-e650f019d05e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:34.889632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:34.892944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4da8f-219f-4a4a-990f-e86f34934129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:34.894289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4da8f-219f-4a4a-990f-e86f34934129 None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:34.894912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4da8f-219f-4a4a-990f-e86f34934129 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:34.895505 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4da8f-219f-4a4a-990f-e86f34934129 None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:34.896885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4da8f-219f-4a4a-990f-e86f34934129 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:34.900597 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4da8f-219f-4a4a-990f-e86f34934129 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:34.901960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b746b8-db22-48f5-a589-bf00318d56af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:34.902723 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b746b8-db22-48f5-a589-bf00318d56af None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:34.903020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b746b8-db22-48f5-a589-bf00318d56af None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:34.903300 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b746b8-db22-48f5-a589-bf00318d56af None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:34.903992 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b746b8-db22-48f5-a589-bf00318d56af None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:34.906035 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b746b8-db22-48f5-a589-bf00318d56af None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:34.918047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.sql.core [None req-00b746b8-db22-48f5-a589-bf00318d56af None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 22:09:34.918047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 23 22:09:34.918047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [parameters: {'id': '1f5d5696ed434eff88061107f634b5d7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 23 22:09:34.918047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81680) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 22:09:34.944012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-00b746b8-db22-48f5-a589-bf00318d56af None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 23 22:09:34.944012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:34.944012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 22:09:34.944012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 22:09:34.944012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 22:09:34.944012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 22:09:34.944012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 22:09:34.944012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 22:09:34.944012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 22:09:34.944012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application conn.query(q) Aug 23 22:09:34.944012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 22:09:34.944012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 22:09:34.944012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 22:09:34.944012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application result.read() Aug 23 22:09:34.944012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return ref.to_dict() Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 22:09:34.945918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application next(self.gen) Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application yield resource Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application next(self.gen) Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise self.value Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application session.commit() Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 22:09:34.947960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self.session.flush() Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application compat.raise_( Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 22:09:34.950643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 22:09:34.959978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 22:09:34.959978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 22:09:34.959978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 23 22:09:34.959978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 23 22:09:34.959978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 22:09:34.959978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 22:09:34.959978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 22:09:34.959978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 22:09:34.959978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 22:09:34.959978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 22:09:34.959978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 22:09:34.959978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 22:09:34.959978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 22:09:34.959978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 22:09:34.959978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 22:09:34.959978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception Aug 23 22:09:34.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 22:09:34.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 22:09:34.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 22:09:34.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 22:09:34.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 22:09:34.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 22:09:34.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 22:09:34.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application conn.query(q) Aug 23 22:09:34.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 22:09:34.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 22:09:34.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 22:09:34.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application result.read() Aug 23 22:09:34.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 22:09:34.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 22:09:34.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 22:09:34.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 22:09:34.963353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 22:09:34.963353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 22:09:34.963353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 22:09:34.963353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 22:09:34.963353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 22:09:34.963353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 23 22:09:34.963353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [parameters: {'id': '1f5d5696ed434eff88061107f634b5d7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 23 22:09:34.963353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 22:09:34.963353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:34.963353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:09:34.963353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:34.963353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:34.963353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:34.963353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:34.963353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:34.963353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:34.964916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:34.964916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:34.964916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:34.964916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:34.964916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:34.964916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:34.964916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 23 22:09:34.964916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 23 22:09:34.964916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:34.964916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:34.964916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 23 22:09:34.964916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 23 22:09:34.964916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 22:09:34.964916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 22:09:34.964916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 23 22:09:34.964916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:34.967923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 240/477] 10.222.0.88 () {60 vars in 1131 bytes} [Tue Aug 23 22:09:34 2022] POST /identity/v3/roles => generated 144 bytes in 246 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:34.967923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b746b8-db22-48f5-a589-bf00318d56af None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:34.968269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 238/478] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:09:34 2022] POST /identity/v3/users => generated 249 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:09:35.167944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b746b8-db22-48f5-a589-bf00318d56af None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:35.237323 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccf515-2c9f-4378-947e-2e542d6a0982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:35.238868 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccf515-2c9f-4378-947e-2e542d6a0982 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:35.239624 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccf515-2c9f-4378-947e-2e542d6a0982 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:35.240365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccf515-2c9f-4378-947e-2e542d6a0982 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:35.261570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccf515-2c9f-4378-947e-2e542d6a0982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:35.273720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-02ccf515-2c9f-4378-947e-2e542d6a0982 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:35.273720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:35.273720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:35.273720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:35.273720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:35.273720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:35.273720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:35.273720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:35.273720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:35.273720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:35.273720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:35.273720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:35.273720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:35.273720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:35.273720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:35.273720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:35.275519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:35.275519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:35.275519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:35.275519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:35.275519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:35.275519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:35.275519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:35.275519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:35.275519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:35.275519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:35.275519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:35.275519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:35.275519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:35.275519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:35.275519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:35.275519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:35.277203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:35.277203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:35.277203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:35.277203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:35.277203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:35.277203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:35.277203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:35.277203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:35.277203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:35.277203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:35.277203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:35.277203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:35.280986 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 241/479] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:09:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:35.288282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4da8f-219f-4a4a-990f-e86f34934129 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:35.315156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:35.483015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4da8f-219f-4a4a-990f-e86f34934129 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:35.554608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d46caf-6f8d-47b6-9485-f3227bafdf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:35.555765 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d46caf-6f8d-47b6-9485-f3227bafdf54 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:35.556236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d46caf-6f8d-47b6-9485-f3227bafdf54 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:35.556685 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d46caf-6f8d-47b6-9485-f3227bafdf54 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:35.557989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d46caf-6f8d-47b6-9485-f3227bafdf54 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:35.558527 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d46caf-6f8d-47b6-9485-f3227bafdf54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:35.561171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d46caf-6f8d-47b6-9485-f3227bafdf54 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:35.578363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 239/480] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:37.256410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfaee6-8f27-4a2c-b653-b846340f388b None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:37.257495 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfaee6-8f27-4a2c-b653-b846340f388b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:37.258024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfaee6-8f27-4a2c-b653-b846340f388b None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:37.259950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 242/481] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:37.266193 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe90944-89c1-45eb-9a04-358a4cdff303 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:37.267323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe90944-89c1-45eb-9a04-358a4cdff303 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:37.267959 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe90944-89c1-45eb-9a04-358a4cdff303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:37.288402 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-dbe90944-89c1-45eb-9a04-358a4cdff303 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:37.290098 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-dbe90944-89c1-45eb-9a04-358a4cdff303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:37.412231 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 240/482] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:37.838105 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe9049-a96a-43f8-b5fb-cf9ee22993f7 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:37.839123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe9049-a96a-43f8-b5fb-cf9ee22993f7 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:37.839928 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe9049-a96a-43f8-b5fb-cf9ee22993f7 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:37.843348 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 243/483] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:37.851044 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815686bc-65c5-4ab6-baff-bdcb6f639e66 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:37.851966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815686bc-65c5-4ab6-baff-bdcb6f639e66 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:37.852733 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815686bc-65c5-4ab6-baff-bdcb6f639e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:37.872822 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:37.886213 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-815686bc-65c5-4ab6-baff-bdcb6f639e66 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:37.887870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-815686bc-65c5-4ab6-baff-bdcb6f639e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:37.994570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe9049-a96a-43f8-b5fb-cf9ee22993f7 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:38.039903 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 241/484] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:38.080263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:38.196726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe9049-a96a-43f8-b5fb-cf9ee22993f7 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:38.266279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9029981-d006-4d4f-a21b-ff3a6193cccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:38.269393 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9029981-d006-4d4f-a21b-ff3a6193cccf None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:38.270187 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9029981-d006-4d4f-a21b-ff3a6193cccf None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:38.270953 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9029981-d006-4d4f-a21b-ff3a6193cccf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:38.291917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9029981-d006-4d4f-a21b-ff3a6193cccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:38.303026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-a9029981-d006-4d4f-a21b-ff3a6193cccf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:38.303026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:38.303026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:38.303026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:38.303026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:38.303026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:38.303026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:38.303026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:38.303026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:38.303026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:38.303026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:38.303026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:38.303026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:38.303026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:38.303026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:38.303026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:38.304785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:38.304785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:38.304785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:38.304785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:38.304785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:38.304785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:38.304785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:38.304785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:38.304785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:38.304785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:38.304785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:38.304785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:38.304785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:38.304785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:38.304785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:38.304785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:38.306885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:38.306885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:38.306885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:38.306885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:38.306885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:38.306885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:38.306885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:38.306885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:38.306885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:38.306885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:38.306885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:38.306885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:38.311205 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 244/485] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:09:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:09:38.326262 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815686bc-65c5-4ab6-baff-bdcb6f639e66 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:38.463657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815686bc-65c5-4ab6-baff-bdcb6f639e66 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:38.500107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3441c-8008-4c18-a566-301767df0181 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:38.500523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3441c-8008-4c18-a566-301767df0181 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:38.500668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3441c-8008-4c18-a566-301767df0181 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:38.507983 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 245/486] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:38.509390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc319a-76b3-4b1b-a78e-c2bc8d09401a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:38.509676 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc319a-76b3-4b1b-a78e-c2bc8d09401a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:38.509957 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc319a-76b3-4b1b-a78e-c2bc8d09401a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:38.510636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0f3f4-a15e-487c-8c13-1d1be032c5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:38.511266 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0f3f4-a15e-487c-8c13-1d1be032c5f3 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:38.511591 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0f3f4-a15e-487c-8c13-1d1be032c5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:38.511861 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0f3f4-a15e-487c-8c13-1d1be032c5f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:38.512617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0f3f4-a15e-487c-8c13-1d1be032c5f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:38.512924 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0f3f4-a15e-487c-8c13-1d1be032c5f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:38.514568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0f3f4-a15e-487c-8c13-1d1be032c5f3 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:38.517927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:38.531567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 242/487] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:09:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:38.542830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0f3f4-a15e-487c-8c13-1d1be032c5f3 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:38.551136 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-4cfc319a-76b3-4b1b-a78e-c2bc8d09401a None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:38.552397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4cfc319a-76b3-4b1b-a78e-c2bc8d09401a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:38.655654 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 246/488] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:38.659216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555a03b-a1ee-42e8-97a7-07b34f6a0ab9 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:38.660036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555a03b-a1ee-42e8-97a7-07b34f6a0ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:38.660797 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555a03b-a1ee-42e8-97a7-07b34f6a0ab9 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:38.665181 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 247/489] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:38.673491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943cacb-05e5-4dc1-ba2a-ddc3697e83a8 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:38.674445 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943cacb-05e5-4dc1-ba2a-ddc3697e83a8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:38.675526 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943cacb-05e5-4dc1-ba2a-ddc3697e83a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:38.712941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-8943cacb-05e5-4dc1-ba2a-ddc3697e83a8 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:38.713413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8943cacb-05e5-4dc1-ba2a-ddc3697e83a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:38.774248 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0f3f4-a15e-487c-8c13-1d1be032c5f3 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:38.805407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:38.858312 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d909e1d-a31e-4057-90f6-c1230a01ad59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:38.859769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d909e1d-a31e-4057-90f6-c1230a01ad59 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:38.859935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d909e1d-a31e-4057-90f6-c1230a01ad59 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:38.860357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d909e1d-a31e-4057-90f6-c1230a01ad59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:38.881263 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d909e1d-a31e-4057-90f6-c1230a01ad59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:38.884100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d909e1d-a31e-4057-90f6-c1230a01ad59 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:38.913467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-2d909e1d-a31e-4057-90f6-c1230a01ad59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:38.913467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:38.913467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:38.913467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:38.913467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:38.913467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:38.913467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:38.913467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:38.913467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:38.913467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:38.913467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:38.913467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:38.913467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:38.913467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:38.913467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:38.913467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:38.915192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:38.916889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:38.916889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:38.916889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:38.918601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 248/491] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:38.918866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 243/491] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:38.929127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d909e1d-a31e-4057-90f6-c1230a01ad59 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:38.933220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943cacb-05e5-4dc1-ba2a-ddc3697e83a8 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:39.123834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:39.216493 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943cacb-05e5-4dc1-ba2a-ddc3697e83a8 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:39.249247 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d909e1d-a31e-4057-90f6-c1230a01ad59 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:39.287337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e831545-93c4-464b-83f4-326448aa4052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:39.288136 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e831545-93c4-464b-83f4-326448aa4052 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:39.288448 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e831545-93c4-464b-83f4-326448aa4052 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:39.288689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e831545-93c4-464b-83f4-326448aa4052 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:39.289383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e831545-93c4-464b-83f4-326448aa4052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:39.289961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e831545-93c4-464b-83f4-326448aa4052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:39.291212 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e831545-93c4-464b-83f4-326448aa4052 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:39.310681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 249/492] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:39.310681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e831545-93c4-464b-83f4-326448aa4052 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:39.311950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414dd57-d095-4081-92c9-5f3cf1ee21cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:39.312636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414dd57-d095-4081-92c9-5f3cf1ee21cc None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:39.312909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414dd57-d095-4081-92c9-5f3cf1ee21cc None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:39.313220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414dd57-d095-4081-92c9-5f3cf1ee21cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:39.326504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414dd57-d095-4081-92c9-5f3cf1ee21cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:39.332022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-d414dd57-d095-4081-92c9-5f3cf1ee21cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:39.332022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:39.332022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:39.332022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:39.332022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:39.332022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:39.332022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:39.332022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:39.332022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:39.332022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:39.332022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:39.332022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:39.332022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:39.332022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:39.332022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:39.332022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:39.334163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:39.334163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:39.334163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:39.334163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:39.334163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:39.334163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:39.334163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:39.334163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:39.334163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:39.334163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:39.334163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:39.334163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:39.334163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:39.334163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:39.334163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:39.334163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:39.335455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:39.335455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:39.335455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:39.335455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:39.335455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:39.335455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:39.335455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:39.335455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:39.335455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:39.335455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:39.335455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:39.335455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:39.335455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 244/493] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:09:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:39.335455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b3ef2-0ae0-41f0-8e64-745103e5714c None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:39.336186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b3ef2-0ae0-41f0-8e64-745103e5714c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:39.336186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b3ef2-0ae0-41f0-8e64-745103e5714c None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:39.337431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 245/494] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:39.337869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b3ef2-0ae0-41f0-8e64-745103e5714c None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:39.347695 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:39.470095 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b3ef2-0ae0-41f0-8e64-745103e5714c None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:39.479394 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e831545-93c4-464b-83f4-326448aa4052 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:39.483585 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:39.518057 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bf1ab-627a-4545-8a4e-ef5b4a4f9f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:39.518799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bf1ab-627a-4545-8a4e-ef5b4a4f9f96 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:39.519081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bf1ab-627a-4545-8a4e-ef5b4a4f9f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:39.519371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bf1ab-627a-4545-8a4e-ef5b4a4f9f96 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:39.529640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547bf1ab-627a-4545-8a4e-ef5b4a4f9f96 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:39.531125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547bf1ab-627a-4545-8a4e-ef5b4a4f9f96 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:39.538754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c796c6f-ffe2-4b54-a6ca-10318cf362b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:39.539864 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c796c6f-ffe2-4b54-a6ca-10318cf362b6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:39.540333 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c796c6f-ffe2-4b54-a6ca-10318cf362b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:39.540772 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c796c6f-ffe2-4b54-a6ca-10318cf362b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:39.545525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-547bf1ab-627a-4545-8a4e-ef5b4a4f9f96 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 22:09:39.545525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:39.545525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:39.545525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:39.545525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:39.545525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:39.545525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:39.545525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:39.545525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:39.545525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:39.545525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:39.545525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:39.545525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:09:39.545525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:09:39.545525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:09:39.545525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:39.546669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:39.548667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:39.548667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:39.548667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:09:39.548667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:09:39.548667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:09:39.548667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:09:39.548667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:09:39.548667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:09:39.548667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 22:09:39.548667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:39.548667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 246/495] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:39.548667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bf1ab-627a-4545-8a4e-ef5b4a4f9f96 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:39.570642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c796c6f-ffe2-4b54-a6ca-10318cf362b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:39.581785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-0c796c6f-ffe2-4b54-a6ca-10318cf362b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:39.581785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:39.581785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:39.581785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:39.581785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:39.581785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:39.581785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:39.581785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:39.581785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:39.581785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:39.581785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:39.581785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:39.581785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:39.581785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:39.581785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:39.581785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:39.583357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:39.583357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:39.583357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:39.583357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:39.583357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:39.583357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:39.583357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:39.583357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:39.583357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:39.583357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:39.583357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:39.583357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:39.583357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:39.583357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:39.583357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:39.583357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:39.584179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:39.584179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:39.584179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:39.584179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:39.584179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:39.584179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:39.584179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:39.584179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:39.584179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:39.584179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:39.584179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:39.584179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:39.584179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 250/496] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:09:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:39.584904 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6579d-7acc-49ef-86b3-24ec27dad20e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:39.584982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6579d-7acc-49ef-86b3-24ec27dad20e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:39.585352 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6579d-7acc-49ef-86b3-24ec27dad20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:39.611717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b3f6579d-7acc-49ef-86b3-24ec27dad20e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:39.613611 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b3f6579d-7acc-49ef-86b3-24ec27dad20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:39.715181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bf1ab-627a-4545-8a4e-ef5b4a4f9f96 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:39.716331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 251/497] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:39.718293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6579d-7acc-49ef-86b3-24ec27dad20e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:39.747810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:39.773409 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9892c-2777-402b-b0d2-e0f43f7ceecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:39.775383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9892c-2777-402b-b0d2-e0f43f7ceecc None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:39.775383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9892c-2777-402b-b0d2-e0f43f7ceecc None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:39.775383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9892c-2777-402b-b0d2-e0f43f7ceecc None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:39.775923 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9892c-2777-402b-b0d2-e0f43f7ceecc None admin] RBAC: Adding query filter params (name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:39.775923 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9892c-2777-402b-b0d2-e0f43f7ceecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:39.777498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9892c-2777-402b-b0d2-e0f43f7ceecc None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:39.795316 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 247/498] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:09:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:39.795316 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9892c-2777-402b-b0d2-e0f43f7ceecc None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:39.858911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:39.912501 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6579d-7acc-49ef-86b3-24ec27dad20e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:39.979372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4eea5-ad4f-4f4a-812e-8748533716af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:39.981146 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4eea5-ad4f-4f4a-812e-8748533716af None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:39.982008 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4eea5-ad4f-4f4a-812e-8748533716af None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:39.982511 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4eea5-ad4f-4f4a-812e-8748533716af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:39.996225 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9892c-2777-402b-b0d2-e0f43f7ceecc None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:40.006351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4eea5-ad4f-4f4a-812e-8748533716af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:40.016658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-90f4eea5-ad4f-4f4a-812e-8748533716af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:40.016658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:40.016658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:40.016658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:40.016658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:40.016658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:40.016658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:40.016658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:40.016658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:40.016658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:40.016658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:40.016658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:40.016658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:40.016658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:40.016658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:40.016658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:40.018733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:40.018733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:40.018733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:40.018733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:40.018733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:40.018733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:40.018733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:40.018733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:40.018733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:40.018733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:40.018733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:40.018733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:40.018733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:40.018733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:40.018733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:40.018733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:40.021172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:40.021172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:40.021172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:40.021172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:40.021172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:40.021172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:40.021172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:40.021172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:40.021172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:40.021172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:40.021172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:40.021172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:40.021172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 252/499] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:09:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:40.021172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4eea5-ad4f-4f4a-812e-8748533716af None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:40.058590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8bbed-1ecf-495a-88aa-3c2ae649068d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:40.059277 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8bbed-1ecf-495a-88aa-3c2ae649068d None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:40.059590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8bbed-1ecf-495a-88aa-3c2ae649068d None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:40.059903 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8bbed-1ecf-495a-88aa-3c2ae649068d None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:40.061215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8bbed-1ecf-495a-88aa-3c2ae649068d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:40.061588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8bbed-1ecf-495a-88aa-3c2ae649068d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:40.063095 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8bbed-1ecf-495a-88aa-3c2ae649068d None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:40.106949 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 248/500] 10.222.0.88 () {56 vars in 1137 bytes} [Tue Aug 23 22:09:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:40.107818 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8bbed-1ecf-495a-88aa-3c2ae649068d None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:40.126536 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:40.167319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4eea5-ad4f-4f4a-812e-8748533716af None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:40.206297 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18890a7a-9600-4185-a519-fe9739528698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:40.207020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18890a7a-9600-4185-a519-fe9739528698 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:40.207247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18890a7a-9600-4185-a519-fe9739528698 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:40.207527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18890a7a-9600-4185-a519-fe9739528698 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:40.208229 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18890a7a-9600-4185-a519-fe9739528698 None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:40.208537 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18890a7a-9600-4185-a519-fe9739528698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:40.210060 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18890a7a-9600-4185-a519-fe9739528698 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:40.220951 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 253/501] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:09:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:40.222745 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18890a7a-9600-4185-a519-fe9739528698 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:40.250252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:40.268622 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8bbed-1ecf-495a-88aa-3c2ae649068d None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:40.322387 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b06fee-381e-49fb-985e-eedfbbd99e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:40.323037 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b06fee-381e-49fb-985e-eedfbbd99e65 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:40.323320 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b06fee-381e-49fb-985e-eedfbbd99e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:40.323596 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b06fee-381e-49fb-985e-eedfbbd99e65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:40.336696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b06fee-381e-49fb-985e-eedfbbd99e65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:40.339198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b06fee-381e-49fb-985e-eedfbbd99e65 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:40.361617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-d6b06fee-381e-49fb-985e-eedfbbd99e65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:40.361617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:40.361617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:40.361617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:40.361617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:40.361617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:40.361617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:40.361617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:40.361617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:40.361617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:40.361617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:40.361617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:40.361617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:40.361617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:40.361617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:40.361617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:40.363553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:40.365528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:40.365528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:40.365528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:40.365528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 249/502] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:40.365528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b06fee-381e-49fb-985e-eedfbbd99e65 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:40.374735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18890a7a-9600-4185-a519-fe9739528698 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:40.434121 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45342656-c940-40b1-b8de-a0db2ab2765b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:40.435130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45342656-c940-40b1-b8de-a0db2ab2765b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:40.435624 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45342656-c940-40b1-b8de-a0db2ab2765b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:40.436108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45342656-c940-40b1-b8de-a0db2ab2765b None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:40.437329 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45342656-c940-40b1-b8de-a0db2ab2765b None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:40.437842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45342656-c940-40b1-b8de-a0db2ab2765b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:40.441247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45342656-c940-40b1-b8de-a0db2ab2765b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:40.443872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:40.457563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 254/503] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:09:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:40.459195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45342656-c940-40b1-b8de-a0db2ab2765b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:40.500460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b06fee-381e-49fb-985e-eedfbbd99e65 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:40.559540 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62fc6d-1cf5-471d-98d6-2724afb3ccd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:40.560087 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62fc6d-1cf5-471d-98d6-2724afb3ccd7 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:40.560367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62fc6d-1cf5-471d-98d6-2724afb3ccd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:40.560691 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62fc6d-1cf5-471d-98d6-2724afb3ccd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:40.578786 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62fc6d-1cf5-471d-98d6-2724afb3ccd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:40.582172 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62fc6d-1cf5-471d-98d6-2724afb3ccd7 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:40.585714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:40.594633 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45342656-c940-40b1-b8de-a0db2ab2765b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:40.610513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-7b62fc6d-1cf5-471d-98d6-2724afb3ccd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:40.610513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:40.610513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:40.610513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:40.610513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:40.610513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:40.610513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:40.610513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:40.610513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:40.610513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:40.610513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:40.610513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:40.610513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:40.610513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:40.610513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:40.610513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:40.612463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:40.614040 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:40.614040 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:40.614040 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:40.614040 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 250/504] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:40.615190 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62fc6d-1cf5-471d-98d6-2724afb3ccd7 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:40.638278 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42a264-2e6a-47c8-a64e-e08df88d1ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:40.638904 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42a264-2e6a-47c8-a64e-e08df88d1ade None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:40.639182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42a264-2e6a-47c8-a64e-e08df88d1ade None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:40.639459 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42a264-2e6a-47c8-a64e-e08df88d1ade None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:40.720795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:40.733359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42a264-2e6a-47c8-a64e-e08df88d1ade None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:40.736802 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42a264-2e6a-47c8-a64e-e08df88d1ade None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:40.757134 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62fc6d-1cf5-471d-98d6-2724afb3ccd7 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:40.801011 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e2be4-cc9a-48a0-aa57-bea8fcb4ca31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:40.801692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e2be4-cc9a-48a0-aa57-bea8fcb4ca31 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:40.802004 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e2be4-cc9a-48a0-aa57-bea8fcb4ca31 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:40.802285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e2be4-cc9a-48a0-aa57-bea8fcb4ca31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:40.815511 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e2be4-cc9a-48a0-aa57-bea8fcb4ca31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:40.816921 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e2be4-cc9a-48a0-aa57-bea8fcb4ca31 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:40.832793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-898e2be4-cc9a-48a0-aa57-bea8fcb4ca31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:40.832793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:40.832793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:40.832793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:40.832793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:40.832793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:40.832793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:40.832793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:40.832793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:40.832793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:40.832793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:40.832793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:40.832793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:40.832793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:40.832793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:40.832793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:40.834494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:40.836162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:40.836162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:40.836162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:40.836162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 251/505] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:40.836162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e2be4-cc9a-48a0-aa57-bea8fcb4ca31 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:40.893682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 255/506] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:40.895119 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42a264-2e6a-47c8-a64e-e08df88d1ade None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:40.903837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:40.980379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e2be4-cc9a-48a0-aa57-bea8fcb4ca31 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:41.030940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646306e-ffad-408c-bf04-6e7d8689aa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:41.031718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646306e-ffad-408c-bf04-6e7d8689aa10 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:41.032064 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646306e-ffad-408c-bf04-6e7d8689aa10 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:41.032567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646306e-ffad-408c-bf04-6e7d8689aa10 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:41.033736 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:41.034964 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646306e-ffad-408c-bf04-6e7d8689aa10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:41.035514 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646306e-ffad-408c-bf04-6e7d8689aa10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:41.037636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646306e-ffad-408c-bf04-6e7d8689aa10 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:41.041559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42a264-2e6a-47c8-a64e-e08df88d1ade None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:41.053206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 252/507] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:41.054452 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646306e-ffad-408c-bf04-6e7d8689aa10 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:41.097423 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd0f87-c334-4bd2-8496-f82d1e9dc933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:41.098410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd0f87-c334-4bd2-8496-f82d1e9dc933 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:41.098905 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd0f87-c334-4bd2-8496-f82d1e9dc933 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:41.099389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd0f87-c334-4bd2-8496-f82d1e9dc933 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:41.118113 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd0f87-c334-4bd2-8496-f82d1e9dc933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:41.120643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd0f87-c334-4bd2-8496-f82d1e9dc933 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:41.146941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-c8cd0f87-c334-4bd2-8496-f82d1e9dc933 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:41.146941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:41.146941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:41.146941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:41.146941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:41.146941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:41.146941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:41.146941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:41.146941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:41.146941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:41.146941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:41.146941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:41.146941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:41.146941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:41.146941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:41.146941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:41.148000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:41.148856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:41.148856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:41.148856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:41.148856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 256/508] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:41.150215 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd0f87-c334-4bd2-8496-f82d1e9dc933 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:41.183299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646306e-ffad-408c-bf04-6e7d8689aa10 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:41.193157 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:41.231730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27607a-5f04-4cde-a7a4-1c07e15b02fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:41.232521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27607a-5f04-4cde-a7a4-1c07e15b02fa None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:41.232904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27607a-5f04-4cde-a7a4-1c07e15b02fa None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:41.233309 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27607a-5f04-4cde-a7a4-1c07e15b02fa None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:41.234085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27607a-5f04-4cde-a7a4-1c07e15b02fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:41.234497 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27607a-5f04-4cde-a7a4-1c07e15b02fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:41.236920 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27607a-5f04-4cde-a7a4-1c07e15b02fa None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:41.250617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 253/509] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:41.252413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27607a-5f04-4cde-a7a4-1c07e15b02fa None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:41.306027 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd0f87-c334-4bd2-8496-f82d1e9dc933 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:41.367520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa1793-2300-46e1-8f75-fbbebd7da543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:41.368454 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa1793-2300-46e1-8f75-fbbebd7da543 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:41.368952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa1793-2300-46e1-8f75-fbbebd7da543 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:41.369465 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa1793-2300-46e1-8f75-fbbebd7da543 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:41.370603 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa1793-2300-46e1-8f75-fbbebd7da543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:41.371134 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa1793-2300-46e1-8f75-fbbebd7da543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:41.373765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa1793-2300-46e1-8f75-fbbebd7da543 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:41.375195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27607a-5f04-4cde-a7a4-1c07e15b02fa None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:41.396574 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 257/510] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:41.397955 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa1793-2300-46e1-8f75-fbbebd7da543 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:41.416051 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:41.420168 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7408360-3dc0-4b3b-b5f4-83205fcdd307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:41.420820 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7408360-3dc0-4b3b-b5f4-83205fcdd307 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:41.421127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7408360-3dc0-4b3b-b5f4-83205fcdd307 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:41.421395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7408360-3dc0-4b3b-b5f4-83205fcdd307 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:41.433126 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7408360-3dc0-4b3b-b5f4-83205fcdd307 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:41.434620 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7408360-3dc0-4b3b-b5f4-83205fcdd307 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:41.450603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-b7408360-3dc0-4b3b-b5f4-83205fcdd307 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:41.450603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:41.450603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:41.450603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:41.450603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:41.450603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:41.450603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:41.450603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:41.450603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:41.450603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:41.450603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:41.450603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:41.450603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:41.450603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:41.450603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:41.450603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:41.452501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:41.454313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:41.454313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:41.454313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:41.454313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 254/511] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:41.454313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7408360-3dc0-4b3b-b5f4-83205fcdd307 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:41.494182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:41.559876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa1793-2300-46e1-8f75-fbbebd7da543 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:41.595323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7408360-3dc0-4b3b-b5f4-83205fcdd307 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:41.601890 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82e605-d19a-42be-b14e-0801ca88cdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:41.602865 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82e605-d19a-42be-b14e-0801ca88cdfb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:41.603317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82e605-d19a-42be-b14e-0801ca88cdfb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:41.603758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82e605-d19a-42be-b14e-0801ca88cdfb None admin] PATH_INFO: `/v3/users/glance` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:41.616650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82e605-d19a-42be-b14e-0801ca88cdfb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:41.619101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82e605-d19a-42be-b14e-0801ca88cdfb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:41.639919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-0c82e605-d19a-42be-b14e-0801ca88cdfb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 22:09:41.639919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:41.639919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:41.639919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:41.639919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:41.639919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:41.639919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:41.639919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:41.639919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:41.639919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:41.639919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:41.639919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:41.639919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:09:41.639919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:09:41.639919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:09:41.639919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:41.641682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:41.644012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:41.644012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:41.644012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:09:41.644012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:09:41.644012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:09:41.644012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:09:41.644012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:09:41.644012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:09:41.644012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 22:09:41.644012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:41.644012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 258/512] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:41.644012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82e605-d19a-42be-b14e-0801ca88cdfb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:41.655701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f46f92-8376-474a-b370-66a73955b07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:41.656664 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f46f92-8376-474a-b370-66a73955b07e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:41.656988 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f46f92-8376-474a-b370-66a73955b07e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:41.657441 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f46f92-8376-474a-b370-66a73955b07e None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:41.659119 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f46f92-8376-474a-b370-66a73955b07e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:41.659643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f46f92-8376-474a-b370-66a73955b07e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:41.661950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f46f92-8376-474a-b370-66a73955b07e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:41.680188 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 255/513] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:41.681097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f46f92-8376-474a-b370-66a73955b07e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:41.749329 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:41.783000 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:41.799690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82e605-d19a-42be-b14e-0801ca88cdfb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:41.818533 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f46f92-8376-474a-b370-66a73955b07e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:41.853546 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b213579-9109-4124-9227-c54ae0a40f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:41.854500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b213579-9109-4124-9227-c54ae0a40f76 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:41.854836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b213579-9109-4124-9227-c54ae0a40f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:41.855156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b213579-9109-4124-9227-c54ae0a40f76 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:41.855893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b213579-9109-4124-9227-c54ae0a40f76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:41.857764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b213579-9109-4124-9227-c54ae0a40f76 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:41.870642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebb747-6353-48fc-9f7b-ed56c05fe37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:41.871501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebb747-6353-48fc-9f7b-ed56c05fe37c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:41.871966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebb747-6353-48fc-9f7b-ed56c05fe37c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:41.872441 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebb747-6353-48fc-9f7b-ed56c05fe37c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:41.883326 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebb747-6353-48fc-9f7b-ed56c05fe37c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:41.884815 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebb747-6353-48fc-9f7b-ed56c05fe37c None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:41.911515 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 259/514] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:09:41 2022] POST /identity/v3/users => generated 247 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:09:41.911776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-fbebb747-6353-48fc-9f7b-ed56c05fe37c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:09:41.911776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:41.911776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:41.911776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:41.911776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:41.911776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:41.911776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:41.911776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:41.911776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:41.911776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:41.911776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:41.911776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:41.911776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:09:41.911776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:09:41.911776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:09:41.911776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:41.913449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:41.915135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:41.915135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:41.915135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:09:41.915135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:09:41.915135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:09:41.915135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:09:41.915135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:09:41.915135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:09:41.915135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:09:41.915135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:41.916179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b213579-9109-4124-9227-c54ae0a40f76 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:41.916310 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 256/515] 10.222.0.88 () {56 vars in 1093 bytes} [Tue Aug 23 22:09:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:41.916310 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebb747-6353-48fc-9f7b-ed56c05fe37c None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:42.070081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b213579-9109-4124-9227-c54ae0a40f76 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:42.105451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebb747-6353-48fc-9f7b-ed56c05fe37c None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:42.115906 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:42.147109 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242bd23b-743c-480c-8e6a-a3e364b7bf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:42.147999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bd23b-743c-480c-8e6a-a3e364b7bf5a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.148354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bd23b-743c-480c-8e6a-a3e364b7bf5a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.148690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bd23b-743c-480c-8e6a-a3e364b7bf5a None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.149563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242bd23b-743c-480c-8e6a-a3e364b7bf5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:42.150451 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242bd23b-743c-480c-8e6a-a3e364b7bf5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:42.151358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242bd23b-743c-480c-8e6a-a3e364b7bf5a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:42.167166 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 260/516] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:42.168573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155773f-77bf-4ba9-a659-66417792139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:42.168770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242bd23b-743c-480c-8e6a-a3e364b7bf5a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:42.170097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155773f-77bf-4ba9-a659-66417792139c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.170810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155773f-77bf-4ba9-a659-66417792139c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.171530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155773f-77bf-4ba9-a659-66417792139c None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.174198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155773f-77bf-4ba9-a659-66417792139c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:42.174742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155773f-77bf-4ba9-a659-66417792139c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:42.177436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155773f-77bf-4ba9-a659-66417792139c None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:42.194088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:42.211279 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 257/517] 10.222.0.88 () {56 vars in 1137 bytes} [Tue Aug 23 22:09:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:42.212545 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155773f-77bf-4ba9-a659-66417792139c None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:42.360142 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242bd23b-743c-480c-8e6a-a3e364b7bf5a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:42.372473 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155773f-77bf-4ba9-a659-66417792139c None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:42.416668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69180e48-22a0-4989-82b1-f96820b60835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:42.417449 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69180e48-22a0-4989-82b1-f96820b60835 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.417781 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69180e48-22a0-4989-82b1-f96820b60835 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.418061 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69180e48-22a0-4989-82b1-f96820b60835 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.427402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69180e48-22a0-4989-82b1-f96820b60835 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:42.429000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69180e48-22a0-4989-82b1-f96820b60835 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:42.442160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642eb50-f48a-4ae3-8565-e566671f60bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:42.443203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642eb50-f48a-4ae3-8565-e566671f60bd None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.443695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642eb50-f48a-4ae3-8565-e566671f60bd None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.444180 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642eb50-f48a-4ae3-8565-e566671f60bd None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.444846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-69180e48-22a0-4989-82b1-f96820b60835 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 22:09:42.444846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:42.444846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:42.444846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:42.444846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:42.444846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:42.444846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:42.444846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:42.444846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:42.444846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:42.444846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:42.444846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:42.444846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:09:42.444846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:09:42.444846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:09:42.444846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:42.445771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:42.446663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:42.446663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:42.446663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:09:42.446663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:09:42.446663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:09:42.446663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:09:42.446663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:09:42.446663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:09:42.446663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 22:09:42.446663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:42.447684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642eb50-f48a-4ae3-8565-e566671f60bd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:42.447684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642eb50-f48a-4ae3-8565-e566671f60bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:42.447684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642eb50-f48a-4ae3-8565-e566671f60bd None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:42.450724 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 261/518] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:09:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:42.451483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69180e48-22a0-4989-82b1-f96820b60835 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:42.471845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 258/519] 10.222.0.88 () {56 vars in 1133 bytes} [Tue Aug 23 22:09:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:42.473288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642eb50-f48a-4ae3-8565-e566671f60bd None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:42.511246 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:42.566326 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:42.586838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69180e48-22a0-4989-82b1-f96820b60835 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:42.632596 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642eb50-f48a-4ae3-8565-e566671f60bd None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:42.634338 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82fd63-b0eb-422e-a16a-4c99afadf798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:42.635441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82fd63-b0eb-422e-a16a-4c99afadf798 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.635963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82fd63-b0eb-422e-a16a-4c99afadf798 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.636525 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82fd63-b0eb-422e-a16a-4c99afadf798 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.646039 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82fd63-b0eb-422e-a16a-4c99afadf798 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:42.647794 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82fd63-b0eb-422e-a16a-4c99afadf798 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:42.670666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-df82fd63-b0eb-422e-a16a-4c99afadf798 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:42.670666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:42.670666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:42.670666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:42.670666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:42.670666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:42.670666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:42.670666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:42.670666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:42.670666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:42.670666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:42.670666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:42.670666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:09:42.670666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:09:42.670666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:42.672551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:42.674529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:09:42.674529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:09:42.674529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:09:42.674529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:09:42.674529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:09:42.674529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:09:42.674529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:42.674529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:42.678096 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 262/520] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:42.680025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82fd63-b0eb-422e-a16a-4c99afadf798 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:42.698774 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa6c11-15b8-41a3-ba6d-2f38bf46931f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:42.699691 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa6c11-15b8-41a3-ba6d-2f38bf46931f None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.700156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa6c11-15b8-41a3-ba6d-2f38bf46931f None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.700616 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa6c11-15b8-41a3-ba6d-2f38bf46931f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.820518 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa6c11-15b8-41a3-ba6d-2f38bf46931f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:42.822470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa6c11-15b8-41a3-ba6d-2f38bf46931f None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:42.881035 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82fd63-b0eb-422e-a16a-4c99afadf798 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:42.883903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:42.904921 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:42.929746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e2e30-d197-4f09-8482-638d36bc85a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:42.930337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e2e30-d197-4f09-8482-638d36bc85a4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.930741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e2e30-d197-4f09-8482-638d36bc85a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.931147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e2e30-d197-4f09-8482-638d36bc85a4 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.932026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e2e30-d197-4f09-8482-638d36bc85a4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:42.932450 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e2e30-d197-4f09-8482-638d36bc85a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:42.934143 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e2e30-d197-4f09-8482-638d36bc85a4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:42.980885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 263/521] 10.222.0.88 () {56 vars in 1139 bytes} [Tue Aug 23 22:09:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:42.980885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e2e30-d197-4f09-8482-638d36bc85a4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:42.999502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 259/522] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:42.999502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa6c11-15b8-41a3-ba6d-2f38bf46931f None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:43.118244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e2e30-d197-4f09-8482-638d36bc85a4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:43.144308 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa6c11-15b8-41a3-ba6d-2f38bf46931f None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:43.168120 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22785e33-796f-4c92-87ef-bb57aa2ff7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:43.168918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22785e33-796f-4c92-87ef-bb57aa2ff7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:43.169440 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22785e33-796f-4c92-87ef-bb57aa2ff7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:43.169844 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22785e33-796f-4c92-87ef-bb57aa2ff7a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:43.193469 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a4af3-6345-4350-8c57-2593b603f66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:43.193972 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a4af3-6345-4350-8c57-2593b603f66b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:43.194261 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a4af3-6345-4350-8c57-2593b603f66b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:43.194523 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a4af3-6345-4350-8c57-2593b603f66b None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:43.195383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a4af3-6345-4350-8c57-2593b603f66b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:43.195538 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a4af3-6345-4350-8c57-2593b603f66b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:43.196943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a4af3-6345-4350-8c57-2593b603f66b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:43.210592 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:43.231553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 260/523] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:09:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:43.231553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a4af3-6345-4350-8c57-2593b603f66b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:43.254535 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:43.276332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22785e33-796f-4c92-87ef-bb57aa2ff7a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:43.277874 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22785e33-796f-4c92-87ef-bb57aa2ff7a3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:43.371209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a4af3-6345-4350-8c57-2593b603f66b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:43.420603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdd30b-8239-40d4-afda-f09185986d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:43.421898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdd30b-8239-40d4-afda-f09185986d7a None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:43.422333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdd30b-8239-40d4-afda-f09185986d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:43.422771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdd30b-8239-40d4-afda-f09185986d7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:43.426202 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 264/524] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:43.427465 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22785e33-796f-4c92-87ef-bb57aa2ff7a3 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:43.514941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:43.544887 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdd30b-8239-40d4-afda-f09185986d7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:43.547892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdd30b-8239-40d4-afda-f09185986d7a None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:43.576317 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:43.587840 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22785e33-796f-4c92-87ef-bb57aa2ff7a3 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:43.652787 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98400fc6-d5b1-4c80-aa9a-2c4809d5ba34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:43.654261 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98400fc6-d5b1-4c80-aa9a-2c4809d5ba34 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:43.654982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98400fc6-d5b1-4c80-aa9a-2c4809d5ba34 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:43.655468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98400fc6-d5b1-4c80-aa9a-2c4809d5ba34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:43.677908 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98400fc6-d5b1-4c80-aa9a-2c4809d5ba34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:43.680755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98400fc6-d5b1-4c80-aa9a-2c4809d5ba34 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:43.711698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-98400fc6-d5b1-4c80-aa9a-2c4809d5ba34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:43.711698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:43.711698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:43.711698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:43.711698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:43.711698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:43.711698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:43.711698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:43.711698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:43.711698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:43.711698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:43.711698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:43.711698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:43.711698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:43.711698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:43.711698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:43.713110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:43.714682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:43.714682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:43.714682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:43.718960 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 261/525] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:43.720477 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdd30b-8239-40d4-afda-f09185986d7a None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:43.721637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 265/526] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:43.722850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98400fc6-d5b1-4c80-aa9a-2c4809d5ba34 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:43.859644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdd30b-8239-40d4-afda-f09185986d7a None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:43.880252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:43.889003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98400fc6-d5b1-4c80-aa9a-2c4809d5ba34 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:43.911511 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ba018-4d43-44b2-9ea2-7eb4e2ffd903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:43.912573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ba018-4d43-44b2-9ea2-7eb4e2ffd903 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:43.913064 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ba018-4d43-44b2-9ea2-7eb4e2ffd903 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:43.913553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ba018-4d43-44b2-9ea2-7eb4e2ffd903 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:43.914947 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ba018-4d43-44b2-9ea2-7eb4e2ffd903 None admin] RBAC: Adding query filter params (role.id=23ea214daa734346a6da3a0a8c50d23a, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=7d9ebd88431d46348089bc195491fca5) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:43.915371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ba018-4d43-44b2-9ea2-7eb4e2ffd903 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:43.917583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ba018-4d43-44b2-9ea2-7eb4e2ffd903 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:43.918498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:43.937107 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 262/527] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:43 2022] GET /identity/v3/role_assignments?user.id=7d9ebd88431d46348089bc195491fca5&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=23ea214daa734346a6da3a0a8c50d23a => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:43.937107 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ba018-4d43-44b2-9ea2-7eb4e2ffd903 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:43.958553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825fafa-5582-4f4f-9a3c-ef997200d689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:43.959656 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825fafa-5582-4f4f-9a3c-ef997200d689 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:43.960126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825fafa-5582-4f4f-9a3c-ef997200d689 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:43.960602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825fafa-5582-4f4f-9a3c-ef997200d689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:43.982819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825fafa-5582-4f4f-9a3c-ef997200d689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:43.986327 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825fafa-5582-4f4f-9a3c-ef997200d689 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:44.016175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-4825fafa-5582-4f4f-9a3c-ef997200d689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:44.016175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:44.016175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:44.016175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:44.016175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:44.016175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:44.016175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:44.016175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:44.016175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:44.016175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:44.016175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:44.016175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:44.016175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:44.016175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:44.016175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:44.016175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:44.018426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:44.020058 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:44.020058 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:44.020058 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:44.020058 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 266/528] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:44.021087 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825fafa-5582-4f4f-9a3c-ef997200d689 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:44.131522 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ba018-4d43-44b2-9ea2-7eb4e2ffd903 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:44.184487 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc9ecd-4996-41c9-829f-4fd00124f5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:44.186443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc9ecd-4996-41c9-829f-4fd00124f5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:44.186852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc9ecd-4996-41c9-829f-4fd00124f5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:44.187603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc9ecd-4996-41c9-829f-4fd00124f5d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:44.205967 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc9ecd-4996-41c9-829f-4fd00124f5d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:44.209609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc9ecd-4996-41c9-829f-4fd00124f5d4 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:44.215462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825fafa-5582-4f4f-9a3c-ef997200d689 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:44.242760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-04dc9ecd-4996-41c9-829f-4fd00124f5d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:44.242760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:44.242760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:44.242760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:44.242760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:44.242760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:44.242760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:44.242760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:44.242760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:44.242760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:44.242760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:44.242760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:44.242760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:44.242760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:44.242760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:44.242760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:44.244451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:44.245870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:44.245870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:44.245870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:44.250129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 263/529] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:44.252168 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc9ecd-4996-41c9-829f-4fd00124f5d4 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:44.258960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23df54-dbb3-45e3-9254-56d8edede0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:44.259599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23df54-dbb3-45e3-9254-56d8edede0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:44.259886 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23df54-dbb3-45e3-9254-56d8edede0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:44.260167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23df54-dbb3-45e3-9254-56d8edede0d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:44.260846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23df54-dbb3-45e3-9254-56d8edede0d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:44.261217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23df54-dbb3-45e3-9254-56d8edede0d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:44.262898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23df54-dbb3-45e3-9254-56d8edede0d7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:44.277760 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 267/530] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:44.278439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23df54-dbb3-45e3-9254-56d8edede0d7 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:44.318285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:44.330363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:44.410404 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23df54-dbb3-45e3-9254-56d8edede0d7 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:44.450936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a9b2d-95f5-4fb8-a548-1dab5579d4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:44.451952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a9b2d-95f5-4fb8-a548-1dab5579d4ed None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:44.451952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a9b2d-95f5-4fb8-a548-1dab5579d4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:44.452227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a9b2d-95f5-4fb8-a548-1dab5579d4ed None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:44.452750 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a9b2d-95f5-4fb8-a548-1dab5579d4ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:44.453089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a9b2d-95f5-4fb8-a548-1dab5579d4ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:44.454570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a9b2d-95f5-4fb8-a548-1dab5579d4ed None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:44.469524 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 268/531] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:44.470761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a9b2d-95f5-4fb8-a548-1dab5579d4ed None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:44.480618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc9ecd-4996-41c9-829f-4fd00124f5d4 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:44.531390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ded64-a167-4890-950a-82ac114dbdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:44.532138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ded64-a167-4890-950a-82ac114dbdee None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:44.532811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ded64-a167-4890-950a-82ac114dbdee None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:44.532995 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ded64-a167-4890-950a-82ac114dbdee None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:44.534031 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ded64-a167-4890-950a-82ac114dbdee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:44.534342 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ded64-a167-4890-950a-82ac114dbdee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:44.535814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ded64-a167-4890-950a-82ac114dbdee None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:44.556339 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 264/532] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:44.557414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ded64-a167-4890-950a-82ac114dbdee None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:44.581846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:44.600974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a9b2d-95f5-4fb8-a548-1dab5579d4ed None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:44.644664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d353a-f277-43fb-bb26-78b0105f338b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:44.645482 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d353a-f277-43fb-bb26-78b0105f338b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:44.645764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d353a-f277-43fb-bb26-78b0105f338b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:44.646041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d353a-f277-43fb-bb26-78b0105f338b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:44.654155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d353a-f277-43fb-bb26-78b0105f338b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:44.655682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d353a-f277-43fb-bb26-78b0105f338b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:44.669439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-032d353a-f277-43fb-bb26-78b0105f338b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:44.669439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:44.669439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:44.669439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:44.669439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:44.669439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:44.669439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:44.669439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:44.669439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:44.669439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:44.669439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:44.669439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:44.669439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:09:44.669439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:09:44.669439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:44.670381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:44.671226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:09:44.671226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:09:44.671226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:09:44.671226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:09:44.671226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:09:44.671226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:09:44.671226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:44.671226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:44.675966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 269/533] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:44.676928 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d353a-f277-43fb-bb26-78b0105f338b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:44.707228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:44.718833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ded64-a167-4890-950a-82ac114dbdee None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:44.775688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812e2bd-def1-44a5-ab9a-4993c7986293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:44.777963 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812e2bd-def1-44a5-ab9a-4993c7986293 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:44.778482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812e2bd-def1-44a5-ab9a-4993c7986293 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:44.778971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812e2bd-def1-44a5-ab9a-4993c7986293 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:44.797018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812e2bd-def1-44a5-ab9a-4993c7986293 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:44.800706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812e2bd-def1-44a5-ab9a-4993c7986293 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:44.813976 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d353a-f277-43fb-bb26-78b0105f338b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:44.825471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-9812e2bd-def1-44a5-ab9a-4993c7986293 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:44.825471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:44.825471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:44.825471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:44.825471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:44.825471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:44.825471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:44.825471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:44.825471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:44.825471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:44.825471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:44.825471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:44.825471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:09:44.825471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:09:44.825471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:44.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:44.827249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:09:44.827249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:09:44.827249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:09:44.827249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:09:44.827249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:09:44.827249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:09:44.827249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:44.827249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:44.833498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 265/534] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:44.833498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b772ceb-2327-4bf0-b386-1ae5f0d32965 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:44.833498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b772ceb-2327-4bf0-b386-1ae5f0d32965 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:44.833498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b772ceb-2327-4bf0-b386-1ae5f0d32965 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:44.838379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 266/535] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:44.838379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b772ceb-2327-4bf0-b386-1ae5f0d32965 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:44.858623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc0313-a15d-4afd-9a4f-ff8cac0691d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:44.859298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc0313-a15d-4afd-9a4f-ff8cac0691d0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:44.859577 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc0313-a15d-4afd-9a4f-ff8cac0691d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:44.860294 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc0313-a15d-4afd-9a4f-ff8cac0691d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:44.870237 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc0313-a15d-4afd-9a4f-ff8cac0691d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:44.871663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc0313-a15d-4afd-9a4f-ff8cac0691d0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:44.885365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-c5bc0313-a15d-4afd-9a4f-ff8cac0691d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:44.885365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:44.885365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:44.885365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:44.885365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:44.885365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:44.885365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:44.885365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:44.885365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:44.885365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:44.885365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:44.885365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:44.885365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:09:44.885365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:09:44.885365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:44.886322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:44.887756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:09:44.887756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:09:44.887756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:09:44.887756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:09:44.887756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:09:44.887756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:09:44.887756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:44.887756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:44.887756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 270/536] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:44.887756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc0313-a15d-4afd-9a4f-ff8cac0691d0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:44.898679 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:45.004764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc0313-a15d-4afd-9a4f-ff8cac0691d0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:45.041541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a7391-6c51-4aaf-9c47-22fc005f6068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:45.041785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b772ceb-2327-4bf0-b386-1ae5f0d32965 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:45.042150 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a7391-6c51-4aaf-9c47-22fc005f6068 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:45.042432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a7391-6c51-4aaf-9c47-22fc005f6068 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:45.042704 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a7391-6c51-4aaf-9c47-22fc005f6068 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:45.043404 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a7391-6c51-4aaf-9c47-22fc005f6068 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:45.043711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a7391-6c51-4aaf-9c47-22fc005f6068 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:45.045196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a7391-6c51-4aaf-9c47-22fc005f6068 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:45.067769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 271/537] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:09:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:45.069080 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c13c70-1cae-451f-aa27-86b8b80b6a90 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:45.069450 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c13c70-1cae-451f-aa27-86b8b80b6a90 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:45.069730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c13c70-1cae-451f-aa27-86b8b80b6a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:45.099433 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996b4a7-4f08-431c-a9ef-e1211d7b5d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:45.100028 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f1c13c70-1cae-451f-aa27-86b8b80b6a90 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:45.100138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996b4a7-4f08-431c-a9ef-e1211d7b5d34 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:45.100310 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996b4a7-4f08-431c-a9ef-e1211d7b5d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:45.100580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996b4a7-4f08-431c-a9ef-e1211d7b5d34 None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:45.101311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2996b4a7-4f08-431c-a9ef-e1211d7b5d34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:45.101615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2996b4a7-4f08-431c-a9ef-e1211d7b5d34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:45.102052 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f1c13c70-1cae-451f-aa27-86b8b80b6a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:45.103070 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2996b4a7-4f08-431c-a9ef-e1211d7b5d34 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:45.119740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 267/538] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:09:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:45.120542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996b4a7-4f08-431c-a9ef-e1211d7b5d34 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:45.149541 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:45.194449 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:45.209709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 272/539] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:45.211427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c13c70-1cae-451f-aa27-86b8b80b6a90 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:45.285491 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996b4a7-4f08-431c-a9ef-e1211d7b5d34 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:45.350561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923fe9d-e58c-4ab1-a29e-e114a4cb29ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:45.352085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923fe9d-e58c-4ab1-a29e-e114a4cb29ff None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:45.352840 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923fe9d-e58c-4ab1-a29e-e114a4cb29ff None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:45.353572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923fe9d-e58c-4ab1-a29e-e114a4cb29ff None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:45.355065 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923fe9d-e58c-4ab1-a29e-e114a4cb29ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:45.355844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923fe9d-e58c-4ab1-a29e-e114a4cb29ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:45.358783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923fe9d-e58c-4ab1-a29e-e114a4cb29ff None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:45.388318 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 268/540] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:09:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:45.390256 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923fe9d-e58c-4ab1-a29e-e114a4cb29ff None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:45.413003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c13c70-1cae-451f-aa27-86b8b80b6a90 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:45.491747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4832044-1718-43d7-9814-075180aa3e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:45.492920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832044-1718-43d7-9814-075180aa3e43 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:45.493346 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832044-1718-43d7-9814-075180aa3e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:45.493866 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832044-1718-43d7-9814-075180aa3e43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:45.495252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4832044-1718-43d7-9814-075180aa3e43 None admin] RBAC: Adding query filter params (role.id=23ea214daa734346a6da3a0a8c50d23a, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=6505100494d4496591407c48b48535ee) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:45.495819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4832044-1718-43d7-9814-075180aa3e43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:45.499723 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4832044-1718-43d7-9814-075180aa3e43 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:45.510874 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 273/541] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:45 2022] GET /identity/v3/role_assignments?user.id=6505100494d4496591407c48b48535ee&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=23ea214daa734346a6da3a0a8c50d23a => generated 259 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:45.512382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4832044-1718-43d7-9814-075180aa3e43 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:45.531057 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:45.588911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923fe9d-e58c-4ab1-a29e-e114a4cb29ff None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:45.618659 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:45.677915 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4832044-1718-43d7-9814-075180aa3e43 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:45.686395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab615969-ef6c-4d33-825f-725305104496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:45.687610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab615969-ef6c-4d33-825f-725305104496 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:45.689584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab615969-ef6c-4d33-825f-725305104496 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:45.690371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab615969-ef6c-4d33-825f-725305104496 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:45.692949 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab615969-ef6c-4d33-825f-725305104496 None admin] RBAC: Adding query filter params (role.id=23ea214daa734346a6da3a0a8c50d23a, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=2211e237fe554a82baf129e440a0fb43) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:45.694211 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab615969-ef6c-4d33-825f-725305104496 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:45.697648 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab615969-ef6c-4d33-825f-725305104496 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:45.709005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 269/542] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:45 2022] GET /identity/v3/role_assignments?user.id=2211e237fe554a82baf129e440a0fb43&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=23ea214daa734346a6da3a0a8c50d23a => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:45.712427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab615969-ef6c-4d33-825f-725305104496 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:45.732208 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddbcba-ade6-476e-8cce-a5d2db421abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:45.732839 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddbcba-ade6-476e-8cce-a5d2db421abd None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:45.733153 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddbcba-ade6-476e-8cce-a5d2db421abd None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:45.733434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddbcba-ade6-476e-8cce-a5d2db421abd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:45.734203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddbcba-ade6-476e-8cce-a5d2db421abd None admin] RBAC: Adding query filter params (role.id=23ea214daa734346a6da3a0a8c50d23a, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=e3517b3b077a4a1fa76163657b46930c) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:45.734518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddbcba-ade6-476e-8cce-a5d2db421abd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:45.736394 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddbcba-ade6-476e-8cce-a5d2db421abd None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:45.743866 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 274/543] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:45 2022] GET /identity/v3/role_assignments?user.id=e3517b3b077a4a1fa76163657b46930c&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=23ea214daa734346a6da3a0a8c50d23a => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:45.956311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab615969-ef6c-4d33-825f-725305104496 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:46.034343 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9995bf5-19ad-43da-a18b-9edb8cbaad14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:46.035432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9995bf5-19ad-43da-a18b-9edb8cbaad14 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:46.035931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9995bf5-19ad-43da-a18b-9edb8cbaad14 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:46.036436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9995bf5-19ad-43da-a18b-9edb8cbaad14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:46.040085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:46.069502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9995bf5-19ad-43da-a18b-9edb8cbaad14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:46.080974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-f9995bf5-19ad-43da-a18b-9edb8cbaad14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:46.080974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:46.080974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:46.080974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:46.080974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:46.080974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:46.080974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:46.080974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:46.080974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:46.080974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:46.080974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:46.080974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:46.080974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:46.080974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:46.080974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:46.080974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:46.082007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:46.082007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:46.082007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:46.082007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:46.082007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:46.082007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:46.082007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:46.082007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:46.082007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:46.082007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:46.082007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:46.082007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:46.082007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:46.082007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:46.082007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:46.082007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:46.082789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:46.082789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:46.082789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:46.082789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:46.082789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:46.082789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:46.082789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:46.082789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:46.082789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:46.082789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:46.082789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:46.082789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:46.084020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 270/544] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:09:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:46.091176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddbcba-ade6-476e-8cce-a5d2db421abd None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:46.275622 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddbcba-ade6-476e-8cce-a5d2db421abd None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:46.316266 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:46.335919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ad4ff-ecd6-429b-87f6-883eec554b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:46.336897 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ad4ff-ecd6-429b-87f6-883eec554b75 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:46.337467 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ad4ff-ecd6-429b-87f6-883eec554b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:46.337967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ad4ff-ecd6-429b-87f6-883eec554b75 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:46.338899 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ad4ff-ecd6-429b-87f6-883eec554b75 None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:46.339418 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ad4ff-ecd6-429b-87f6-883eec554b75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:46.341024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ad4ff-ecd6-429b-87f6-883eec554b75 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:46.358193 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 275/545] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:09:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:46.365595 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9995bf5-19ad-43da-a18b-9edb8cbaad14 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:46.578103 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9995bf5-19ad-43da-a18b-9edb8cbaad14 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:46.628009 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cb6d9-2e3a-48a5-94cd-480f361d0c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:46.629482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cb6d9-2e3a-48a5-94cd-480f361d0c4f None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:46.629998 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cb6d9-2e3a-48a5-94cd-480f361d0c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:46.630468 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cb6d9-2e3a-48a5-94cd-480f361d0c4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:46.651172 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82cb6d9-2e3a-48a5-94cd-480f361d0c4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:46.652991 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82cb6d9-2e3a-48a5-94cd-480f361d0c4f None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:46.674237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-a82cb6d9-2e3a-48a5-94cd-480f361d0c4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:46.674237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:46.674237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:46.674237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:46.674237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:46.674237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:46.674237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:46.674237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:46.674237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:46.674237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:46.674237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:46.674237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:46.674237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:46.674237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:46.674237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:46.674237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:46.675204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:46.676114 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:46.676114 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:46.676114 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:46.678080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 271/546] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:46.684138 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ad4ff-ecd6-429b-87f6-883eec554b75 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:46.873340 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ad4ff-ecd6-429b-87f6-883eec554b75 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:46.928173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c746aa0-a438-48f8-b983-fc6323191af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:46.929216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c746aa0-a438-48f8-b983-fc6323191af1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:46.929708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c746aa0-a438-48f8-b983-fc6323191af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:46.930187 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c746aa0-a438-48f8-b983-fc6323191af1 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:46.931071 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c746aa0-a438-48f8-b983-fc6323191af1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:46.932009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c746aa0-a438-48f8-b983-fc6323191af1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:46.933862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c746aa0-a438-48f8-b983-fc6323191af1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:46.953963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 276/547] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:46.960608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cb6d9-2e3a-48a5-94cd-480f361d0c4f None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:47.017463 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57d2c6-f5e9-4fdb-9d22-1f3985a5799f None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:47.018353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57d2c6-f5e9-4fdb-9d22-1f3985a5799f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:47.018842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57d2c6-f5e9-4fdb-9d22-1f3985a5799f None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:47.020928 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 277/548] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:47.026345 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c2958-1591-4627-a10f-9e94fe7f7cb0 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:47.026674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c2958-1591-4627-a10f-9e94fe7f7cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:47.026906 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c2958-1591-4627-a10f-9e94fe7f7cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:47.053389 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:47.059184 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d80c2958-1591-4627-a10f-9e94fe7f7cb0 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:47.061145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d80c2958-1591-4627-a10f-9e94fe7f7cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:47.118546 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:47.151949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 278/549] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:47.179444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cb6d9-2e3a-48a5-94cd-480f361d0c4f None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:47.253812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd51d26-10fd-4638-b4ed-fbe2b279dc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:47.254466 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd51d26-10fd-4638-b4ed-fbe2b279dc64 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:47.254747 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd51d26-10fd-4638-b4ed-fbe2b279dc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:47.255019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd51d26-10fd-4638-b4ed-fbe2b279dc64 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:47.263710 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd51d26-10fd-4638-b4ed-fbe2b279dc64 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:47.265209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd51d26-10fd-4638-b4ed-fbe2b279dc64 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:47.281236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-4bd51d26-10fd-4638-b4ed-fbe2b279dc64 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:09:47.281236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:47.281236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:47.281236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:47.281236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:47.281236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:47.281236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:47.281236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:47.281236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:47.281236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:47.281236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:47.281236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:47.281236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:09:47.281236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:09:47.281236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:09:47.281236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:47.282744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:47.284241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:47.284241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:47.284241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:09:47.284241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:09:47.284241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:09:47.284241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:09:47.284241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:09:47.284241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:09:47.284241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:09:47.284241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:47.284241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 272/550] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:09:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:47.291658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c2958-1591-4627-a10f-9e94fe7f7cb0 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:47.428510 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c2958-1591-4627-a10f-9e94fe7f7cb0 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:47.470582 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323c5d4-e1d1-4e00-b59d-d5eeee9844dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:47.471671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323c5d4-e1d1-4e00-b59d-d5eeee9844dc None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:47.472174 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323c5d4-e1d1-4e00-b59d-d5eeee9844dc None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:47.472650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323c5d4-e1d1-4e00-b59d-d5eeee9844dc None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:47.473652 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323c5d4-e1d1-4e00-b59d-d5eeee9844dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:47.474272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323c5d4-e1d1-4e00-b59d-d5eeee9844dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:47.476047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323c5d4-e1d1-4e00-b59d-d5eeee9844dc None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:47.498327 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 279/551] 10.222.0.88 () {56 vars in 1135 bytes} [Tue Aug 23 22:09:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:47.510332 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd51d26-10fd-4638-b4ed-fbe2b279dc64 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:47.729499 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd51d26-10fd-4638-b4ed-fbe2b279dc64 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:47.742843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323c5d4-e1d1-4e00-b59d-d5eeee9844dc None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:47.765654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:47.775686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ab4bd-5c7f-44bc-b6f6-37077155eb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:47.776287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab4bd-5c7f-44bc-b6f6-37077155eb77 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:47.776565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab4bd-5c7f-44bc-b6f6-37077155eb77 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:47.776833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab4bd-5c7f-44bc-b6f6-37077155eb77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:47.786892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:47.925178 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ab4bd-5c7f-44bc-b6f6-37077155eb77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:47.927945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ab4bd-5c7f-44bc-b6f6-37077155eb77 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:47.937163 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323c5d4-e1d1-4e00-b59d-d5eeee9844dc None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:48.019253 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fbff2-5733-4034-9770-dc9aa7faeee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:48.019818 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fbff2-5733-4034-9770-dc9aa7faeee8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:48.020108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fbff2-5733-4034-9770-dc9aa7faeee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:48.020383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fbff2-5733-4034-9770-dc9aa7faeee8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:48.042811 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365fbff2-5733-4034-9770-dc9aa7faeee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:48.048016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-365fbff2-5733-4034-9770-dc9aa7faeee8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:48.048016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:48.048016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:48.048016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:48.048016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:48.048016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:48.048016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:48.048016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:48.048016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:48.048016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:48.048016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:48.048016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:48.048016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:48.048016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:48.048016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:48.048016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:48.049005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:48.049005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:48.049005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:48.049005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:48.049005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:48.049005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:48.049005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:48.049005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:48.049005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:48.049005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:48.049005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:48.049005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:48.049005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:48.049005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:48.049005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:48.049005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:48.049945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:48.049945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:48.049945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:48.049945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:48.049945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:48.049945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:48.049945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:48.049945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:48.049945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:48.049945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:48.049945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:48.049945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:48.057714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 280/552] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:09:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:48.064153 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fbff2-5733-4034-9770-dc9aa7faeee8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:48.066699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 273/553] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:48.074149 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ab4bd-5c7f-44bc-b6f6-37077155eb77 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:48.087033 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:48.223981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ab4bd-5c7f-44bc-b6f6-37077155eb77 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:48.267171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9d4cd-cb83-40cb-917a-16940d3a268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:48.267825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9d4cd-cb83-40cb-917a-16940d3a268b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:48.268105 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9d4cd-cb83-40cb-917a-16940d3a268b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:48.268375 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9d4cd-cb83-40cb-917a-16940d3a268b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:48.293157 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9d4cd-cb83-40cb-917a-16940d3a268b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:48.294566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9d4cd-cb83-40cb-917a-16940d3a268b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:48.331173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-14f9d4cd-cb83-40cb-917a-16940d3a268b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:48.331173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:48.331173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:48.331173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:48.331173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:48.331173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:48.331173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:48.331173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:48.331173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:48.331173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:48.331173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:48.331173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:48.331173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:48.331173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:48.331173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:48.331173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:48.332289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:48.333245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:48.333245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:48.333245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:48.333245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 274/554] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:48.341841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9d4cd-cb83-40cb-917a-16940d3a268b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:48.370342 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fbff2-5733-4034-9770-dc9aa7faeee8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:48.373272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:48.441854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7685278-95f7-4764-969b-ff1bfda78129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:48.442953 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7685278-95f7-4764-969b-ff1bfda78129 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:48.443443 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7685278-95f7-4764-969b-ff1bfda78129 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:48.443932 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7685278-95f7-4764-969b-ff1bfda78129 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:48.444837 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7685278-95f7-4764-969b-ff1bfda78129 None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:48.453086 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7685278-95f7-4764-969b-ff1bfda78129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:48.456010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7685278-95f7-4764-969b-ff1bfda78129 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:48.459942 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:48.475683 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 281/555] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:09:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:48.481699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7685278-95f7-4764-969b-ff1bfda78129 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:48.509909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9d4cd-cb83-40cb-917a-16940d3a268b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:48.558061 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9483cb8-b4c5-4628-a4dc-ddd9bdb2c438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:48.558607 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9483cb8-b4c5-4628-a4dc-ddd9bdb2c438 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:48.558877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9483cb8-b4c5-4628-a4dc-ddd9bdb2c438 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:48.559147 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9483cb8-b4c5-4628-a4dc-ddd9bdb2c438 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:48.559939 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9483cb8-b4c5-4628-a4dc-ddd9bdb2c438 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:48.560244 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9483cb8-b4c5-4628-a4dc-ddd9bdb2c438 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:48.565104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9483cb8-b4c5-4628-a4dc-ddd9bdb2c438 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:48.577127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 275/556] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:48.578454 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8703f7-1871-4635-b81b-8b6ad77256c5 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:48.578743 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8703f7-1871-4635-b81b-8b6ad77256c5 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:48.579583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8703f7-1871-4635-b81b-8b6ad77256c5 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:48.584388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 276/557] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:48.587430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8703f7-1871-4635-b81b-8b6ad77256c5 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:48.690797 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7685278-95f7-4764-969b-ff1bfda78129 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:48.744026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8703f7-1871-4635-b81b-8b6ad77256c5 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:48.747221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d842a-1270-4ab5-ac67-ffebaa4821ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:48.748025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d842a-1270-4ab5-ac67-ffebaa4821ec None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:48.748532 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d842a-1270-4ab5-ac67-ffebaa4821ec None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:48.748893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d842a-1270-4ab5-ac67-ffebaa4821ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:48.763416 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d842a-1270-4ab5-ac67-ffebaa4821ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:48.764930 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d842a-1270-4ab5-ac67-ffebaa4821ec None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:48.769401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:48.781234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-4d0d842a-1270-4ab5-ac67-ffebaa4821ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:48.781234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:48.781234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:48.781234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:48.781234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:48.781234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:48.781234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:48.781234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:48.781234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:48.781234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:48.781234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:48.781234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:48.781234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:48.781234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:48.781234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:48.781234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:48.782963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:48.783838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:48.783838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:48.783838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:48.783838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 282/558] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:09:48.783838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6786f4-1b1a-468d-a99c-e8e998055091 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:48.784139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6786f4-1b1a-468d-a99c-e8e998055091 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:48.784481 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6786f4-1b1a-468d-a99c-e8e998055091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:48.795799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e1a82-ebba-4612-b3f5-1f365aabd267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:48.796778 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e1a82-ebba-4612-b3f5-1f365aabd267 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:48.796940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e1a82-ebba-4612-b3f5-1f365aabd267 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:48.798304 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e1a82-ebba-4612-b3f5-1f365aabd267 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:48.803913 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9b6786f4-1b1a-468d-a99c-e8e998055091 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:48.805196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9b6786f4-1b1a-468d-a99c-e8e998055091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:48.807486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:48.809952 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e1a82-ebba-4612-b3f5-1f365aabd267 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:48.811579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e1a82-ebba-4612-b3f5-1f365aabd267 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:48.826605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-a40e1a82-ebba-4612-b3f5-1f365aabd267 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:48.826605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:48.826605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:48.826605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:48.826605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:48.826605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:48.826605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:48.826605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:48.826605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:48.826605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:48.826605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:48.826605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:48.826605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:09:48.826605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:09:48.826605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:48.835673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:48.837702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:09:48.837702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:09:48.837702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:09:48.837702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:09:48.837702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:09:48.837702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:09:48.837702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:48.837702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:48.837702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 277/559] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:48.837702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24551337-5294-4686-821a-28645cf0f4b0 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:48.837702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24551337-5294-4686-821a-28645cf0f4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:48.838507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24551337-5294-4686-821a-28645cf0f4b0 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:48.838507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 278/560] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:48.838507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee1576-d6b6-4f72-84cc-0a185dc9dbc8 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:48.838507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee1576-d6b6-4f72-84cc-0a185dc9dbc8 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:48.838507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee1576-d6b6-4f72-84cc-0a185dc9dbc8 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:48.838507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 279/561] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:48.838507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee1576-d6b6-4f72-84cc-0a185dc9dbc8 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:48.895263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 283/562] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:48.896405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246953cd-ef75-4f19-8fe4-fd94338270db None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:48.896800 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246953cd-ef75-4f19-8fe4-fd94338270db None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:48.896962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246953cd-ef75-4f19-8fe4-fd94338270db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:48.926185 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-246953cd-ef75-4f19-8fe4-fd94338270db None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:48.928056 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-246953cd-ef75-4f19-8fe4-fd94338270db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:49.000327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee1576-d6b6-4f72-84cc-0a185dc9dbc8 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:49.013448 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 284/563] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:49.014441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246953cd-ef75-4f19-8fe4-fd94338270db None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:49.078646 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1400fc-488d-4c0c-897f-9575585d2470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:49.081429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1400fc-488d-4c0c-897f-9575585d2470 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:49.082210 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1400fc-488d-4c0c-897f-9575585d2470 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:49.082994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1400fc-488d-4c0c-897f-9575585d2470 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:49.084381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1400fc-488d-4c0c-897f-9575585d2470 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:49.085665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1400fc-488d-4c0c-897f-9575585d2470 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:49.088074 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1400fc-488d-4c0c-897f-9575585d2470 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:49.117588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 280/564] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:49.119926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a834b-77fd-4d69-851c-21f2fb6a9404 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:49.120283 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a834b-77fd-4d69-851c-21f2fb6a9404 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:49.120756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a834b-77fd-4d69-851c-21f2fb6a9404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:49.143777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:49.155976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-565a834b-77fd-4d69-851c-21f2fb6a9404 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:49.158720 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-565a834b-77fd-4d69-851c-21f2fb6a9404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:49.213234 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:49.216182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246953cd-ef75-4f19-8fe4-fd94338270db None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:49.269840 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 281/565] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:49.270940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a834b-77fd-4d69-851c-21f2fb6a9404 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:49.284595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d5ac0-319b-49f5-9b76-8f69115804fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:49.285422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d5ac0-319b-49f5-9b76-8f69115804fa None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:49.285787 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d5ac0-319b-49f5-9b76-8f69115804fa None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:49.286132 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d5ac0-319b-49f5-9b76-8f69115804fa None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:49.287027 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d5ac0-319b-49f5-9b76-8f69115804fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:49.287407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d5ac0-319b-49f5-9b76-8f69115804fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:49.289653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d5ac0-319b-49f5-9b76-8f69115804fa None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:49.304743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 285/566] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:09:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:49.306357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d5ac0-319b-49f5-9b76-8f69115804fa None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:49.410814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a834b-77fd-4d69-851c-21f2fb6a9404 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:49.461613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d5ac0-319b-49f5-9b76-8f69115804fa None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:49.472485 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab9bad-a243-4532-a0c7-aafe43f95cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:49.473828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab9bad-a243-4532-a0c7-aafe43f95cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:49.474112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab9bad-a243-4532-a0c7-aafe43f95cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:49.474384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab9bad-a243-4532-a0c7-aafe43f95cd6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:49.483125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab9bad-a243-4532-a0c7-aafe43f95cd6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:49.484653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab9bad-a243-4532-a0c7-aafe43f95cd6 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:49.501503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:49.508127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-82ab9bad-a243-4532-a0c7-aafe43f95cd6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 22:09:49.508127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:49.508127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:49.508127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:49.508127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:49.508127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:49.508127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:49.508127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:49.508127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:49.508127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:49.508127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:49.508127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:49.508127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:09:49.508127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:09:49.508127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:09:49.508127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:49.509414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:49.510658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:49.510658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:49.510658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:09:49.510658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:09:49.510658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:09:49.510658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:09:49.510658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:09:49.510658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:09:49.510658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 22:09:49.510658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:49.510658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 282/567] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:49.511483 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab9bad-a243-4532-a0c7-aafe43f95cd6 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:49.521439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdc987-6641-4803-b7da-babd857c1321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:49.522105 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdc987-6641-4803-b7da-babd857c1321 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:49.522394 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdc987-6641-4803-b7da-babd857c1321 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:49.522831 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdc987-6641-4803-b7da-babd857c1321 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:49.539243 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdc987-6641-4803-b7da-babd857c1321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:49.546466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-4bfdc987-6641-4803-b7da-babd857c1321 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:49.546466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:49.546466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:49.546466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:49.546466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:49.546466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:49.546466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:49.546466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:49.546466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:49.546466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:49.546466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:49.546466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:49.546466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:49.546466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:49.546466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:49.546466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:49.548129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:49.548129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:49.548129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:49.548129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:49.548129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:49.548129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:49.548129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:49.548129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:49.548129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:49.548129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:49.548129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:49.548129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:49.548129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:49.548129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:49.548129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:49.548129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:49.549151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:49.549151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:49.549151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:49.549151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:49.549151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:49.549151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:49.549151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:49.549151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:49.549151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:49.549151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:49.549151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:49.549151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:49.549151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 286/568] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:09:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:49.549912 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdc987-6641-4803-b7da-babd857c1321 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:49.553685 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:49.642201 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab9bad-a243-4532-a0c7-aafe43f95cd6 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:49.665476 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdc987-6641-4803-b7da-babd857c1321 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:49.677241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8c80b-4d65-4010-84b2-0089ff152c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:49.677851 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8c80b-4d65-4010-84b2-0089ff152c28 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:49.678128 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8c80b-4d65-4010-84b2-0089ff152c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:49.678398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8c80b-4d65-4010-84b2-0089ff152c28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:49.679122 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8c80b-4d65-4010-84b2-0089ff152c28 None admin] RBAC: Adding query filter params (role.id=23ea214daa734346a6da3a0a8c50d23a, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=601a3defca70428689ee22517d7bcb19) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:49.679426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8c80b-4d65-4010-84b2-0089ff152c28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:49.680866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8c80b-4d65-4010-84b2-0089ff152c28 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:49.686721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 283/569] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:49 2022] GET /identity/v3/role_assignments?user.id=601a3defca70428689ee22517d7bcb19&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=23ea214daa734346a6da3a0a8c50d23a => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:49.687685 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8c80b-4d65-4010-84b2-0089ff152c28 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:49.706988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3847a0-4c42-48a1-8750-4203716023dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:49.707969 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3847a0-4c42-48a1-8750-4203716023dc None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:49.708336 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3847a0-4c42-48a1-8750-4203716023dc None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:49.708860 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3847a0-4c42-48a1-8750-4203716023dc None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:49.710155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3847a0-4c42-48a1-8750-4203716023dc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:49.710810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3847a0-4c42-48a1-8750-4203716023dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:49.712934 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3847a0-4c42-48a1-8750-4203716023dc None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:49.742044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 287/570] 10.222.0.88 () {56 vars in 1137 bytes} [Tue Aug 23 22:09:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:49.742044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3847a0-4c42-48a1-8750-4203716023dc None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:49.834382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:49.846546 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:49.858956 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8c80b-4d65-4010-84b2-0089ff152c28 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:49.905011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3847a0-4c42-48a1-8750-4203716023dc None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:49.929223 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bb0e6-fc11-4291-b312-daed8f129bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:49.930363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bb0e6-fc11-4291-b312-daed8f129bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:49.930870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bb0e6-fc11-4291-b312-daed8f129bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:49.931450 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bb0e6-fc11-4291-b312-daed8f129bd9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:49.945311 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db791cf-723b-4926-9090-927154d11f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:49.946037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db791cf-723b-4926-9090-927154d11f49 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:49.946221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db791cf-723b-4926-9090-927154d11f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:49.946463 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db791cf-723b-4926-9090-927154d11f49 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:49.947129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db791cf-723b-4926-9090-927154d11f49 None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:49.947425 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db791cf-723b-4926-9090-927154d11f49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:49.948790 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db791cf-723b-4926-9090-927154d11f49 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:49.954574 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bb0e6-fc11-4291-b312-daed8f129bd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:49.960944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-8d2bb0e6-fc11-4291-b312-daed8f129bd9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:49.960944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:49.960944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:49.960944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:49.960944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:49.960944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:49.960944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:49.960944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:49.960944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:49.960944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:49.960944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:49.960944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:49.960944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:49.960944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:49.960944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:49.960944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:49.962810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:49.962810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:49.962810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:49.962810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:49.962810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:49.962810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:49.962810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:49.962810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:49.962810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:49.962810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:49.962810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:49.962810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:49.962810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:49.962810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:49.962810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:49.962810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:49.964410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:49.964410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:49.964410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:49.964410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:49.964410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:49.964410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:49.964410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:49.964410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:49.964410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:49.964410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:49.964410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:49.964410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:49.964410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 284/571] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:09:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:49.964410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bb0e6-fc11-4291-b312-daed8f129bd9 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:49.969338 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 288/572] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:09:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:49.971338 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db791cf-723b-4926-9090-927154d11f49 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:50.130027 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db791cf-723b-4926-9090-927154d11f49 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:50.140687 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bb0e6-fc11-4291-b312-daed8f129bd9 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:50.146733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:50.185399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838602cb-4971-46a9-8808-b739023448f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:50.186016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838602cb-4971-46a9-8808-b739023448f8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:50.186358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838602cb-4971-46a9-8808-b739023448f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:50.186634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838602cb-4971-46a9-8808-b739023448f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:50.195498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d62b7b-3437-4a67-a5c0-0c87b2d77570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:50.196895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d62b7b-3437-4a67-a5c0-0c87b2d77570 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:50.197283 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d62b7b-3437-4a67-a5c0-0c87b2d77570 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:50.197622 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d62b7b-3437-4a67-a5c0-0c87b2d77570 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:50.200549 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838602cb-4971-46a9-8808-b739023448f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:50.202107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838602cb-4971-46a9-8808-b739023448f8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:50.212402 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d62b7b-3437-4a67-a5c0-0c87b2d77570 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:50.218198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-b3d62b7b-3437-4a67-a5c0-0c87b2d77570 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:50.218198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:50.218198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:50.218198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:50.218198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:50.218198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:50.218198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:50.218198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:50.218198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:50.218198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:50.218198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:50.218198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:50.218198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:50.218198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:50.218198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:50.218198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:50.220834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:50.220834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:50.220834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:50.220834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:50.220834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:50.220834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:50.220834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:50.220834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:50.220834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:50.220834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:50.220834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:50.220834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:50.220834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:50.220834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:50.220834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:50.220834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:50.223932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:50.223932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:50.223932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:50.223932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:50.223932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:50.223932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:50.223932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:50.223932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:50.223932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:50.223932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:50.223932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:50.223932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:50.223932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 285/573] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:09:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:50.223932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d62b7b-3437-4a67-a5c0-0c87b2d77570 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:50.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-838602cb-4971-46a9-8808-b739023448f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:50.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:50.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:50.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:50.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:50.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:50.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:50.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:50.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:50.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:50.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:50.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:50.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:50.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:50.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:50.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:50.233503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:50.235221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:50.235221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:50.235221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:50.235221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 289/574] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:50.235221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838602cb-4971-46a9-8808-b739023448f8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:50.252968 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:50.369189 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838602cb-4971-46a9-8808-b739023448f8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:50.389646 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d62b7b-3437-4a67-a5c0-0c87b2d77570 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:50.411925 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ee28e-9491-4235-97e5-802594715d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:50.412752 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ee28e-9491-4235-97e5-802594715d16 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:50.413114 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ee28e-9491-4235-97e5-802594715d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:50.413472 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ee28e-9491-4235-97e5-802594715d16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:50.427200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ee28e-9491-4235-97e5-802594715d16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:50.428939 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ee28e-9491-4235-97e5-802594715d16 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:50.440839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbf452-1b76-4469-be12-1eb7df8457f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:50.442123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbf452-1b76-4469-be12-1eb7df8457f5 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:50.442650 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbf452-1b76-4469-be12-1eb7df8457f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:50.443113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbf452-1b76-4469-be12-1eb7df8457f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:50.443974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbf452-1b76-4469-be12-1eb7df8457f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:50.444478 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbf452-1b76-4469-be12-1eb7df8457f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:50.446072 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbf452-1b76-4469-be12-1eb7df8457f5 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:50.447232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-827ee28e-9491-4235-97e5-802594715d16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:50.447232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:50.447232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:50.447232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:50.447232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:50.447232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:50.447232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:50.447232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:50.447232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:50.447232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:50.447232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:50.447232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:50.447232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:50.447232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:50.447232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:50.447232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:50.448468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:50.449715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:50.449715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:50.449715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:50.449715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 290/575] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:50.450663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ee28e-9491-4235-97e5-802594715d16 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:50.474424 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 286/576] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:09:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:50.474424 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbf452-1b76-4469-be12-1eb7df8457f5 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:50.480322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:50.603701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:50.618365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ee28e-9491-4235-97e5-802594715d16 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:50.641178 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbf452-1b76-4469-be12-1eb7df8457f5 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:50.663308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e024ab4-8992-4b86-8c55-bfd2c2aa1c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:50.664112 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e024ab4-8992-4b86-8c55-bfd2c2aa1c3e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:50.664589 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e024ab4-8992-4b86-8c55-bfd2c2aa1c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:50.665086 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e024ab4-8992-4b86-8c55-bfd2c2aa1c3e None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:50.665911 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e024ab4-8992-4b86-8c55-bfd2c2aa1c3e None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:50.666337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e024ab4-8992-4b86-8c55-bfd2c2aa1c3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:50.668204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e024ab4-8992-4b86-8c55-bfd2c2aa1c3e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:50.684958 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 291/577] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:09:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:50.685932 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e024ab4-8992-4b86-8c55-bfd2c2aa1c3e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:50.700392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39603881-2912-402a-ade9-15fb6c877f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:50.701402 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39603881-2912-402a-ade9-15fb6c877f5d None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:50.701871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39603881-2912-402a-ade9-15fb6c877f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:50.702336 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39603881-2912-402a-ade9-15fb6c877f5d None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:50.703479 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39603881-2912-402a-ade9-15fb6c877f5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:50.704514 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39603881-2912-402a-ade9-15fb6c877f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:50.706589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39603881-2912-402a-ade9-15fb6c877f5d None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:50.723559 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 287/578] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:50.724519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39603881-2912-402a-ade9-15fb6c877f5d None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:50.825168 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:50.862265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e024ab4-8992-4b86-8c55-bfd2c2aa1c3e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:50.906904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39603881-2912-402a-ade9-15fb6c877f5d None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:50.916444 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd2280-017a-422b-b1ef-5ca2f7a47af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:50.917192 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd2280-017a-422b-b1ef-5ca2f7a47af8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:50.917526 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd2280-017a-422b-b1ef-5ca2f7a47af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:50.917810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd2280-017a-422b-b1ef-5ca2f7a47af8 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:50.918660 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd2280-017a-422b-b1ef-5ca2f7a47af8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:50.919044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd2280-017a-422b-b1ef-5ca2f7a47af8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:50.920988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd2280-017a-422b-b1ef-5ca2f7a47af8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:50.935997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 292/579] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:50.937013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd2280-017a-422b-b1ef-5ca2f7a47af8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:50.977777 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3de69b-8f6f-4e5d-ae0d-9c42d547416c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:50.978782 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3de69b-8f6f-4e5d-ae0d-9c42d547416c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:50.979276 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3de69b-8f6f-4e5d-ae0d-9c42d547416c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:50.979748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3de69b-8f6f-4e5d-ae0d-9c42d547416c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:50.983185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:50.996071 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3de69b-8f6f-4e5d-ae0d-9c42d547416c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:50.997694 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3de69b-8f6f-4e5d-ae0d-9c42d547416c None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:51.015132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-be3de69b-8f6f-4e5d-ae0d-9c42d547416c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:51.015132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:51.015132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:51.015132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:51.015132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:51.015132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:51.015132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:51.015132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:51.015132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:51.015132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:51.015132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:51.015132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:51.015132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:51.015132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:51.015132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:51.015132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:51.017740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:51.019529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:51.019529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:51.019529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:51.019529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 288/580] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:51.019529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3de69b-8f6f-4e5d-ae0d-9c42d547416c None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:51.093635 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd2280-017a-422b-b1ef-5ca2f7a47af8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:51.147268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f645c-9da1-456d-a0cc-ca5cfb160c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:51.147924 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f645c-9da1-456d-a0cc-ca5cfb160c89 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:51.148357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f645c-9da1-456d-a0cc-ca5cfb160c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:51.148638 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f645c-9da1-456d-a0cc-ca5cfb160c89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:51.151552 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:51.163693 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3de69b-8f6f-4e5d-ae0d-9c42d547416c None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:51.165131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f645c-9da1-456d-a0cc-ca5cfb160c89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:51.166731 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f645c-9da1-456d-a0cc-ca5cfb160c89 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:51.185217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-cd7f645c-9da1-456d-a0cc-ca5cfb160c89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:51.185217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:51.185217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:51.185217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:51.185217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:51.185217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:51.185217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:51.185217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:51.185217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:51.185217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:51.185217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:51.185217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:51.185217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:51.185217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:51.185217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:51.185217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:51.186585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:51.187594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:51.187594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:51.187594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:51.187594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 293/581] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:51.187594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f645c-9da1-456d-a0cc-ca5cfb160c89 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:51.225150 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74de5e7-00c6-40e8-ba63-0fba6ee75e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:51.225955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74de5e7-00c6-40e8-ba63-0fba6ee75e4e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:51.226369 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74de5e7-00c6-40e8-ba63-0fba6ee75e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:51.226819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74de5e7-00c6-40e8-ba63-0fba6ee75e4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:51.235871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74de5e7-00c6-40e8-ba63-0fba6ee75e4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:51.237317 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74de5e7-00c6-40e8-ba63-0fba6ee75e4e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:51.251357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-a74de5e7-00c6-40e8-ba63-0fba6ee75e4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:51.251357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:51.251357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:51.251357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:51.251357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:51.251357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:51.251357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:51.251357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:51.251357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:51.251357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:51.251357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:51.251357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:51.251357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:09:51.251357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:09:51.251357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:51.253154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:51.255109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:09:51.255109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:09:51.255109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:09:51.255109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:09:51.255109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:09:51.255109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:09:51.255109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:51.255109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:51.257473 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 289/582] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:51.258443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74de5e7-00c6-40e8-ba63-0fba6ee75e4e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:51.325692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:51.342952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f645c-9da1-456d-a0cc-ca5cfb160c89 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:51.389425 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e05beb-af77-412e-9da7-a4886657207b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:51.390157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e05beb-af77-412e-9da7-a4886657207b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:51.390512 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e05beb-af77-412e-9da7-a4886657207b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:51.390863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e05beb-af77-412e-9da7-a4886657207b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:51.400537 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e05beb-af77-412e-9da7-a4886657207b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:51.402046 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e05beb-af77-412e-9da7-a4886657207b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:51.410840 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74de5e7-00c6-40e8-ba63-0fba6ee75e4e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:51.414947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-74e05beb-af77-412e-9da7-a4886657207b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:09:51.414947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:51.414947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:51.414947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:51.414947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:51.414947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:51.414947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:51.414947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:51.414947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:51.414947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:51.414947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:51.414947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:51.414947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:09:51.414947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:09:51.414947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:09:51.414947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:51.416571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:51.419073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:51.419073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:51.419073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:09:51.419073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:09:51.419073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:09:51.419073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:09:51.419073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:09:51.419073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:09:51.419073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:09:51.419073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:51.420769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 294/583] 10.222.0.88 () {56 vars in 1093 bytes} [Tue Aug 23 22:09:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:51.421647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e05beb-af77-412e-9da7-a4886657207b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:51.467053 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60ba49-9982-4e39-bfea-abc0e1ac292e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:51.468085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60ba49-9982-4e39-bfea-abc0e1ac292e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:51.468558 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60ba49-9982-4e39-bfea-abc0e1ac292e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:51.469010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60ba49-9982-4e39-bfea-abc0e1ac292e None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:51.470549 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60ba49-9982-4e39-bfea-abc0e1ac292e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:51.471084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60ba49-9982-4e39-bfea-abc0e1ac292e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:51.472725 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60ba49-9982-4e39-bfea-abc0e1ac292e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:51.489155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 290/584] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:51.490135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60ba49-9982-4e39-bfea-abc0e1ac292e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:51.498089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:51.574686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e05beb-af77-412e-9da7-a4886657207b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:51.629754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6704ac1-2acb-4c7c-94f9-036251712e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:51.630716 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6704ac1-2acb-4c7c-94f9-036251712e0b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:51.631168 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6704ac1-2acb-4c7c-94f9-036251712e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:51.631590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6704ac1-2acb-4c7c-94f9-036251712e0b None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:51.632694 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6704ac1-2acb-4c7c-94f9-036251712e0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:51.633225 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6704ac1-2acb-4c7c-94f9-036251712e0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:51.635399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6704ac1-2acb-4c7c-94f9-036251712e0b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:51.661441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 295/585] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:51.661441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6704ac1-2acb-4c7c-94f9-036251712e0b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:51.665949 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60ba49-9982-4e39-bfea-abc0e1ac292e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:51.677872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:51.719450 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e94191-d6c1-4406-8564-8964c60aa992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:51.720616 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e94191-d6c1-4406-8564-8964c60aa992 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:51.720893 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e94191-d6c1-4406-8564-8964c60aa992 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:51.721382 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e94191-d6c1-4406-8564-8964c60aa992 None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:51.722098 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e94191-d6c1-4406-8564-8964c60aa992 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:51.722406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e94191-d6c1-4406-8564-8964c60aa992 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:51.723868 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e94191-d6c1-4406-8564-8964c60aa992 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:51.739668 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 291/586] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:09:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:51.740450 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e94191-d6c1-4406-8564-8964c60aa992 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:51.790162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6704ac1-2acb-4c7c-94f9-036251712e0b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:51.801148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:51.845976 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354d9e7-49dd-420b-a57d-85eccadcd9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:51.848653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354d9e7-49dd-420b-a57d-85eccadcd9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:51.848653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354d9e7-49dd-420b-a57d-85eccadcd9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:51.848653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354d9e7-49dd-420b-a57d-85eccadcd9e4 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:51.848653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354d9e7-49dd-420b-a57d-85eccadcd9e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:51.848653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354d9e7-49dd-420b-a57d-85eccadcd9e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:51.851254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354d9e7-49dd-420b-a57d-85eccadcd9e4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:51.873160 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 296/587] 10.222.0.88 () {56 vars in 1133 bytes} [Tue Aug 23 22:09:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:51.874080 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354d9e7-49dd-420b-a57d-85eccadcd9e4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:51.874351 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e94191-d6c1-4406-8564-8964c60aa992 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:51.924421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d4470-3d8c-4841-b3b1-078ebec5fc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:51.925492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d4470-3d8c-4841-b3b1-078ebec5fc05 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:51.925492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d4470-3d8c-4841-b3b1-078ebec5fc05 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:51.925798 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d4470-3d8c-4841-b3b1-078ebec5fc05 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:51.934291 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d4470-3d8c-4841-b3b1-078ebec5fc05 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:51.935764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d4470-3d8c-4841-b3b1-078ebec5fc05 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:51.948692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-c14d4470-3d8c-4841-b3b1-078ebec5fc05 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 22:09:51.948692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:51.948692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:51.948692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:51.948692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:51.948692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:51.948692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:51.948692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:51.948692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:51.948692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:51.948692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:51.948692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:51.948692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:09:51.948692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:09:51.948692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:09:51.948692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:51.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:51.951983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:51.951983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:51.951983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:09:51.951983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:09:51.951983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:09:51.951983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:09:51.951983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:09:51.951983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:09:51.951983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 22:09:51.951983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:51.951983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 292/588] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:51.951983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1b97d-f62e-4a28-af0d-80d5b367b8b3 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:51.951983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1b97d-f62e-4a28-af0d-80d5b367b8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:51.952868 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1b97d-f62e-4a28-af0d-80d5b367b8b3 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:51.957195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 293/589] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:51.957195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1b97d-f62e-4a28-af0d-80d5b367b8b3 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:51.987699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:52.031036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354d9e7-49dd-420b-a57d-85eccadcd9e4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:52.074620 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b4007-f008-4167-b07f-2af074b3879a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:52.075287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b4007-f008-4167-b07f-2af074b3879a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:52.075577 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b4007-f008-4167-b07f-2af074b3879a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:52.075842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b4007-f008-4167-b07f-2af074b3879a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:52.084524 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b4007-f008-4167-b07f-2af074b3879a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:52.086029 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b4007-f008-4167-b07f-2af074b3879a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:52.099696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-b02b4007-f008-4167-b07f-2af074b3879a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 22:09:52.099696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:52.099696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:52.099696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:52.099696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:52.099696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:52.099696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:52.099696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:52.099696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:52.099696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:52.099696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:52.099696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:52.099696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:09:52.099696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:09:52.099696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:09:52.099696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:52.101964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:52.104223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:52.104223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:52.104223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:09:52.104223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:09:52.104223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:09:52.104223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:09:52.104223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:09:52.104223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:09:52.104223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 22:09:52.104223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:52.104223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 297/590] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:09:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:52.104223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b4007-f008-4167-b07f-2af074b3879a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:52.112927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1b97d-f62e-4a28-af0d-80d5b367b8b3 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:52.138476 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:52.165948 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2ace0-2a6a-4427-a8b5-e019dc77f465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:52.167107 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2ace0-2a6a-4427-a8b5-e019dc77f465 None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:52.167592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2ace0-2a6a-4427-a8b5-e019dc77f465 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:52.168102 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2ace0-2a6a-4427-a8b5-e019dc77f465 None admin] PATH_INFO: `/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/7d9ebd88431d46348089bc195491fca5/roles/23ea214daa734346a6da3a0a8c50d23a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:52.196479 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2ace0-2a6a-4427-a8b5-e019dc77f465 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a7317af325f485e8fae49631cb5bc20, user_id=7d9ebd88431d46348089bc195491fca5, role_id=23ea214daa734346a6da3a0a8c50d23a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:52.199254 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2ace0-2a6a-4427-a8b5-e019dc77f465 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:52.218929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 294/591] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:09:51 2022] PUT /identity/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/7d9ebd88431d46348089bc195491fca5/roles/23ea214daa734346a6da3a0a8c50d23a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:52.220878 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2ace0-2a6a-4427-a8b5-e019dc77f465 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:52.234833 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b4007-f008-4167-b07f-2af074b3879a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:52.298625 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad7f84-f098-44e3-b5b9-b32167d96c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:52.299522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad7f84-f098-44e3-b5b9-b32167d96c15 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:52.299939 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad7f84-f098-44e3-b5b9-b32167d96c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:52.300374 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad7f84-f098-44e3-b5b9-b32167d96c15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:52.320702 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad7f84-f098-44e3-b5b9-b32167d96c15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:52.323042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad7f84-f098-44e3-b5b9-b32167d96c15 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:52.341128 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:52.348919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-a4ad7f84-f098-44e3-b5b9-b32167d96c15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:52.348919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:52.348919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:52.348919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:52.348919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:52.348919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:52.348919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:52.348919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:52.348919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:52.348919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:52.348919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:52.348919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:52.348919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:52.348919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:52.348919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:52.348919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:52.350919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:52.353367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:52.353367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:52.353367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:52.359065 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 298/592] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:52.361315 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b3f42-709c-454b-ba2b-d533e4b6ef96 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:52.361784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b3f42-709c-454b-ba2b-d533e4b6ef96 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:52.362340 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b3f42-709c-454b-ba2b-d533e4b6ef96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:52.390549 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-773b3f42-709c-454b-ba2b-d533e4b6ef96 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:52.392429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2ace0-2a6a-4427-a8b5-e019dc77f465 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:52.393006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-773b3f42-709c-454b-ba2b-d533e4b6ef96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:52.449859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326dc68-b52b-435e-b13d-a1dad93ec4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:52.450751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326dc68-b52b-435e-b13d-a1dad93ec4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:52.451248 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326dc68-b52b-435e-b13d-a1dad93ec4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:52.451744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326dc68-b52b-435e-b13d-a1dad93ec4a5 None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:52.453148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326dc68-b52b-435e-b13d-a1dad93ec4a5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:52.453703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326dc68-b52b-435e-b13d-a1dad93ec4a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:52.457550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326dc68-b52b-435e-b13d-a1dad93ec4a5 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:52.488476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 295/593] 10.222.0.88 () {56 vars in 1137 bytes} [Tue Aug 23 22:09:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:52.489657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326dc68-b52b-435e-b13d-a1dad93ec4a5 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:52.518167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 299/594] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:52.519507 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b3f42-709c-454b-ba2b-d533e4b6ef96 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:52.551278 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:52.640237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326dc68-b52b-435e-b13d-a1dad93ec4a5 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:52.692050 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e2ad2-0ffc-4aab-b528-21d2e3a8579f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:52.693253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e2ad2-0ffc-4aab-b528-21d2e3a8579f None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:52.693647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e2ad2-0ffc-4aab-b528-21d2e3a8579f None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:52.693982 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e2ad2-0ffc-4aab-b528-21d2e3a8579f None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:52.694868 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e2ad2-0ffc-4aab-b528-21d2e3a8579f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:52.695302 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e2ad2-0ffc-4aab-b528-21d2e3a8579f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:52.698007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e2ad2-0ffc-4aab-b528-21d2e3a8579f None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:52.701397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:52.713557 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b3f42-709c-454b-ba2b-d533e4b6ef96 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:52.733454 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 296/595] 10.222.0.88 () {56 vars in 1139 bytes} [Tue Aug 23 22:09:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:52.734429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e2ad2-0ffc-4aab-b528-21d2e3a8579f None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:52.765632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f6d41-ad77-41ac-bd50-aa82625f2bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:52.765632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f6d41-ad77-41ac-bd50-aa82625f2bfa None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:52.765632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f6d41-ad77-41ac-bd50-aa82625f2bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:52.765632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f6d41-ad77-41ac-bd50-aa82625f2bfa None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:52.766326 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f6d41-ad77-41ac-bd50-aa82625f2bfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:52.766326 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f6d41-ad77-41ac-bd50-aa82625f2bfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:52.767988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f6d41-ad77-41ac-bd50-aa82625f2bfa None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:52.785048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 300/596] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:52.786013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f6d41-ad77-41ac-bd50-aa82625f2bfa None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:52.883431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:52.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e2ad2-0ffc-4aab-b528-21d2e3a8579f None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:52.933654 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f6d41-ad77-41ac-bd50-aa82625f2bfa None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:52.954758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68f055-3f2a-4d54-863a-5d1cc110906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:52.956036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68f055-3f2a-4d54-863a-5d1cc110906f None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:52.956614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68f055-3f2a-4d54-863a-5d1cc110906f None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:52.957381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68f055-3f2a-4d54-863a-5d1cc110906f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:52.976708 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68f055-3f2a-4d54-863a-5d1cc110906f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:52.979641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68f055-3f2a-4d54-863a-5d1cc110906f None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:52.986488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974976a2-6dd1-4ebd-a3a9-5ae2da8f381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:52.987408 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974976a2-6dd1-4ebd-a3a9-5ae2da8f381d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:52.987849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974976a2-6dd1-4ebd-a3a9-5ae2da8f381d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:52.988331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974976a2-6dd1-4ebd-a3a9-5ae2da8f381d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:52.999769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-ee68f055-3f2a-4d54-863a-5d1cc110906f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:52.999769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:52.999769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:52.999769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:52.999769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:52.999769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:52.999769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:52.999769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:52.999769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:52.999769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:52.999769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:52.999769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:52.999769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:52.999769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:52.999769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:52.999769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:53.001366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:53.002649 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:53.002649 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:53.002649 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:53.002822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974976a2-6dd1-4ebd-a3a9-5ae2da8f381d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:53.002822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974976a2-6dd1-4ebd-a3a9-5ae2da8f381d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:53.005405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 297/597] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:53.006628 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68f055-3f2a-4d54-863a-5d1cc110906f None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:53.021252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-974976a2-6dd1-4ebd-a3a9-5ae2da8f381d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:53.021252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:53.021252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:53.021252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:53.021252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:53.021252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:53.021252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:53.021252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:53.021252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:53.021252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:53.021252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:53.021252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:53.021252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:53.021252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:53.021252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:53.021252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:53.022773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:53.024085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:53.024085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:53.024085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:53.024085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 301/598] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:53.024085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974976a2-6dd1-4ebd-a3a9-5ae2da8f381d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:53.069471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:53.170306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68f055-3f2a-4d54-863a-5d1cc110906f None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:53.224704 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974976a2-6dd1-4ebd-a3a9-5ae2da8f381d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:53.232266 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b9840-43f5-43c1-99b9-88f2a0b46c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:53.233340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b9840-43f5-43c1-99b9-88f2a0b46c2a None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:53.233821 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b9840-43f5-43c1-99b9-88f2a0b46c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:53.234300 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b9840-43f5-43c1-99b9-88f2a0b46c2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:53.248361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b9840-43f5-43c1-99b9-88f2a0b46c2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:53.251725 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:53.251912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b9840-43f5-43c1-99b9-88f2a0b46c2a None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:53.275742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8bb37-4e78-4fa7-9ae0-184c02717ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:53.276554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8bb37-4e78-4fa7-9ae0-184c02717ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:53.276932 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8bb37-4e78-4fa7-9ae0-184c02717ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:53.277353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8bb37-4e78-4fa7-9ae0-184c02717ea5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:53.278897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-838b9840-43f5-43c1-99b9-88f2a0b46c2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:53.278897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:53.278897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:53.278897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:53.278897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:53.278897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:53.278897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:53.278897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:53.278897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:53.278897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:53.278897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:53.278897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:53.278897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:09:53.278897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:09:53.278897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:53.280779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:53.283567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:09:53.283567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:09:53.283567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:09:53.283567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:09:53.283567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:09:53.283567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:09:53.283567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:53.283567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:53.284461 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 298/599] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:53.286470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b9840-43f5-43c1-99b9-88f2a0b46c2a None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:53.304092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c8bb37-4e78-4fa7-9ae0-184c02717ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:53.317161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-59c8bb37-4e78-4fa7-9ae0-184c02717ea5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:53.317161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:53.317161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:53.317161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:53.317161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:53.317161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:53.317161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:53.317161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:53.317161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:53.317161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:53.317161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:53.317161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:53.317161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:53.317161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:53.317161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:53.317161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:53.319268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:53.319268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:53.319268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:53.319268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:53.319268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:53.319268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:53.319268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:53.319268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:53.319268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:53.319268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:53.319268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:53.319268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:53.319268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:53.319268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:53.319268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:53.319268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:53.321127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:53.321127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:53.321127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:53.321127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:53.321127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:53.321127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:53.321127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:53.321127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:53.321127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:53.321127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:53.321127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:53.321127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:53.322335 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 302/600] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:09:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:53.323129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8bb37-4e78-4fa7-9ae0-184c02717ea5 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:53.459002 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b9840-43f5-43c1-99b9-88f2a0b46c2a None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:53.462311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:53.499974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8bb37-4e78-4fa7-9ae0-184c02717ea5 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:53.526662 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ac342-e557-46fe-a066-a83edeb0d7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:53.527719 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ac342-e557-46fe-a066-a83edeb0d7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:53.528196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ac342-e557-46fe-a066-a83edeb0d7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:53.528674 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ac342-e557-46fe-a066-a83edeb0d7f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:53.529888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ac342-e557-46fe-a066-a83edeb0d7f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:53.530433 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ac342-e557-46fe-a066-a83edeb0d7f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:53.533212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ac342-e557-46fe-a066-a83edeb0d7f8 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:53.547553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512669ac-b546-48eb-a193-8151f6b3e465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:53.548250 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512669ac-b546-48eb-a193-8151f6b3e465 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:53.548535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512669ac-b546-48eb-a193-8151f6b3e465 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:53.548817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512669ac-b546-48eb-a193-8151f6b3e465 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:53.549570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512669ac-b546-48eb-a193-8151f6b3e465 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:53.550016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512669ac-b546-48eb-a193-8151f6b3e465 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:53.551812 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512669ac-b546-48eb-a193-8151f6b3e465 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:53.564098 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 299/601] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:53.564098 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ac342-e557-46fe-a066-a83edeb0d7f8 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:53.571642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 303/602] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:53.573769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512669ac-b546-48eb-a193-8151f6b3e465 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:53.630310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:53.743489 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512669ac-b546-48eb-a193-8151f6b3e465 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:53.751972 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ac342-e557-46fe-a066-a83edeb0d7f8 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:53.818008 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc379cf-d4db-444c-93f9-84576a64b8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:53.818954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc379cf-d4db-444c-93f9-84576a64b8cd None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:53.819390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc379cf-d4db-444c-93f9-84576a64b8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:53.819817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc379cf-d4db-444c-93f9-84576a64b8cd None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:53.820401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79e9fd-d7d7-4a36-9011-f82d7ac4745b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:53.820883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc379cf-d4db-444c-93f9-84576a64b8cd None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:53.821387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc379cf-d4db-444c-93f9-84576a64b8cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:53.822523 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79e9fd-d7d7-4a36-9011-f82d7ac4745b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:53.823372 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79e9fd-d7d7-4a36-9011-f82d7ac4745b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:53.823627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc379cf-d4db-444c-93f9-84576a64b8cd None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:53.824415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79e9fd-d7d7-4a36-9011-f82d7ac4745b None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:53.826421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79e9fd-d7d7-4a36-9011-f82d7ac4745b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:53.827875 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79e9fd-d7d7-4a36-9011-f82d7ac4745b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:53.831181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79e9fd-d7d7-4a36-9011-f82d7ac4745b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:53.838402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 304/603] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:09:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:53.839722 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc379cf-d4db-444c-93f9-84576a64b8cd None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:53.861513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 300/604] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:09:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:53.862678 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79e9fd-d7d7-4a36-9011-f82d7ac4745b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:53.916435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:54.039464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:54.045092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc379cf-d4db-444c-93f9-84576a64b8cd None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:54.075135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79e9fd-d7d7-4a36-9011-f82d7ac4745b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:54.115573 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d21e8-636e-4d22-953e-d0bcef1e8002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:54.116793 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d21e8-636e-4d22-953e-d0bcef1e8002 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:54.117321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d21e8-636e-4d22-953e-d0bcef1e8002 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:54.117780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d21e8-636e-4d22-953e-d0bcef1e8002 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:54.132033 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d21e8-636e-4d22-953e-d0bcef1e8002 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:54.134891 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d21e8-636e-4d22-953e-d0bcef1e8002 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:54.141905 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c647d-e577-4b22-82b0-fdde39f7352e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:54.143089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c647d-e577-4b22-82b0-fdde39f7352e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:54.143530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c647d-e577-4b22-82b0-fdde39f7352e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:54.144045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c647d-e577-4b22-82b0-fdde39f7352e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:54.158892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-768d21e8-636e-4d22-953e-d0bcef1e8002 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:54.158892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:54.158892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:54.158892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:54.158892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:54.158892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:54.158892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:54.158892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:54.158892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:54.158892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:54.158892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:54.158892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:54.158892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:09:54.158892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:09:54.158892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:54.160572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:54.170783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:09:54.170783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:09:54.170783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:09:54.170783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:09:54.170783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:09:54.170783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:09:54.170783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:54.170783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:54.171577 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c647d-e577-4b22-82b0-fdde39f7352e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:54.171712 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 305/605] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:54.171856 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c647d-e577-4b22-82b0-fdde39f7352e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:54.172001 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d21e8-636e-4d22-953e-d0bcef1e8002 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:54.192714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-171c647d-e577-4b22-82b0-fdde39f7352e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:54.192714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:54.192714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:54.192714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:54.192714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:54.192714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:54.192714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:54.192714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:54.192714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:54.192714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:54.192714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:54.192714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:54.192714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:09:54.192714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:09:54.192714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:54.195644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:54.197422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:09:54.197422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:09:54.197422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:09:54.197422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:09:54.197422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:09:54.197422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:09:54.197422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:54.197422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:54.197422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 301/606] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:54.197422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c647d-e577-4b22-82b0-fdde39f7352e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:54.352227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d21e8-636e-4d22-953e-d0bcef1e8002 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:54.363833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:54.394283 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c647d-e577-4b22-82b0-fdde39f7352e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:54.415639 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b005a-d63c-43ed-b39a-2857f01ec757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:54.416616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b005a-d63c-43ed-b39a-2857f01ec757 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:54.417041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b005a-d63c-43ed-b39a-2857f01ec757 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:54.417492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b005a-d63c-43ed-b39a-2857f01ec757 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:54.435157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b005a-d63c-43ed-b39a-2857f01ec757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:54.437553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b005a-d63c-43ed-b39a-2857f01ec757 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:54.441633 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a93d5f-cef3-45a5-b53a-e1f67d117554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:54.442349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a93d5f-cef3-45a5-b53a-e1f67d117554 None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:54.442625 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a93d5f-cef3-45a5-b53a-e1f67d117554 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:54.442911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a93d5f-cef3-45a5-b53a-e1f67d117554 None admin] PATH_INFO: `/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/e3517b3b077a4a1fa76163657b46930c/roles/23ea214daa734346a6da3a0a8c50d23a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:54.460794 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a93d5f-cef3-45a5-b53a-e1f67d117554 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a7317af325f485e8fae49631cb5bc20, user_id=e3517b3b077a4a1fa76163657b46930c, role_id=23ea214daa734346a6da3a0a8c50d23a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:54.462291 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a93d5f-cef3-45a5-b53a-e1f67d117554 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:54.463387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-3c6b005a-d63c-43ed-b39a-2857f01ec757 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:54.463387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:54.463387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:54.463387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:54.463387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:54.463387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:54.463387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:54.463387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:54.463387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:54.463387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:54.463387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:54.463387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:54.463387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:54.463387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:54.463387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:54.463387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:54.464967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:54.466473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:54.466473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:54.466473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:54.466473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 306/607] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:54.466473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b005a-d63c-43ed-b39a-2857f01ec757 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:54.494418 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:54.503815 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 302/608] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:09:54 2022] PUT /identity/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/e3517b3b077a4a1fa76163657b46930c/roles/23ea214daa734346a6da3a0a8c50d23a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:54.504942 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a93d5f-cef3-45a5-b53a-e1f67d117554 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:54.696939 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b005a-d63c-43ed-b39a-2857f01ec757 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:54.704966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a93d5f-cef3-45a5-b53a-e1f67d117554 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:54.770379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0b1d9-a70f-4f96-905a-db8e0e6a29a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:54.771283 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0b1d9-a70f-4f96-905a-db8e0e6a29a6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:54.771696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0b1d9-a70f-4f96-905a-db8e0e6a29a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:54.772124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0b1d9-a70f-4f96-905a-db8e0e6a29a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:54.773289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0b1d9-a70f-4f96-905a-db8e0e6a29a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:54.773765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0b1d9-a70f-4f96-905a-db8e0e6a29a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:54.774057 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b640d13-53ae-4889-8058-ebe52c2b0d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:54.774935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b640d13-53ae-4889-8058-ebe52c2b0d1b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:54.775361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b640d13-53ae-4889-8058-ebe52c2b0d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:54.775767 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b640d13-53ae-4889-8058-ebe52c2b0d1b None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:54.776071 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0b1d9-a70f-4f96-905a-db8e0e6a29a6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:54.776874 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b640d13-53ae-4889-8058-ebe52c2b0d1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:54.777362 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b640d13-53ae-4889-8058-ebe52c2b0d1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:54.779664 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b640d13-53ae-4889-8058-ebe52c2b0d1b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:54.782991 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:54.803053 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 303/609] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:09:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:54.809100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 307/610] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:09:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:54.809314 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b640d13-53ae-4889-8058-ebe52c2b0d1b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:54.809714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4455d2-a578-4fa7-9691-ea44b190256a None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:54.810277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4455d2-a578-4fa7-9691-ea44b190256a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:54.810748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4455d2-a578-4fa7-9691-ea44b190256a None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:54.813687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 308/611] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:54.814860 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4455d2-a578-4fa7-9691-ea44b190256a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:54.946143 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:54.998836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4455d2-a578-4fa7-9691-ea44b190256a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:55.000103 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b640d13-53ae-4889-8058-ebe52c2b0d1b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:55.059306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79c80b-c2f3-4583-ad76-1117537a6d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:55.060016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79c80b-c2f3-4583-ad76-1117537a6d15 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:55.060317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79c80b-c2f3-4583-ad76-1117537a6d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:55.060563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79c80b-c2f3-4583-ad76-1117537a6d15 None admin] PATH_INFO: `/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/2211e237fe554a82baf129e440a0fb43/roles/23ea214daa734346a6da3a0a8c50d23a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:55.064559 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33b043-6c93-4371-a424-bf48034af5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:55.065536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33b043-6c93-4371-a424-bf48034af5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:55.065817 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33b043-6c93-4371-a424-bf48034af5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:55.066098 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33b043-6c93-4371-a424-bf48034af5c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:55.066767 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33b043-6c93-4371-a424-bf48034af5c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:55.067075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33b043-6c93-4371-a424-bf48034af5c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:55.068479 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33b043-6c93-4371-a424-bf48034af5c4 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:55.084571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb79c80b-c2f3-4583-ad76-1117537a6d15 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a7317af325f485e8fae49631cb5bc20, user_id=2211e237fe554a82baf129e440a0fb43, role_id=23ea214daa734346a6da3a0a8c50d23a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:55.087103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb79c80b-c2f3-4583-ad76-1117537a6d15 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:55.089659 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 304/612] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:55.090418 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33b043-6c93-4371-a424-bf48034af5c4 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:55.108934 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 309/613] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:09:54 2022] PUT /identity/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/2211e237fe554a82baf129e440a0fb43/roles/23ea214daa734346a6da3a0a8c50d23a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:55.111512 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928acb9b-7fd0-4d3e-8540-134e45c04180 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:55.112055 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928acb9b-7fd0-4d3e-8540-134e45c04180 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:55.112709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928acb9b-7fd0-4d3e-8540-134e45c04180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:55.136905 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-928acb9b-7fd0-4d3e-8540-134e45c04180 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:55.138655 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-928acb9b-7fd0-4d3e-8540-134e45c04180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:55.206865 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:55.223636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 310/614] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:55.224821 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928acb9b-7fd0-4d3e-8540-134e45c04180 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:55.291143 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:55.348757 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928acb9b-7fd0-4d3e-8540-134e45c04180 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:55.378318 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33b043-6c93-4371-a424-bf48034af5c4 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:55.415565 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97188792-438e-4a3f-9aae-2ab00185a58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:55.416632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97188792-438e-4a3f-9aae-2ab00185a58b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:55.417126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97188792-438e-4a3f-9aae-2ab00185a58b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:55.417613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97188792-438e-4a3f-9aae-2ab00185a58b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:55.432545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97188792-438e-4a3f-9aae-2ab00185a58b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:55.434593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97188792-438e-4a3f-9aae-2ab00185a58b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:55.450017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-97188792-438e-4a3f-9aae-2ab00185a58b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:09:55.450017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:55.450017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:55.450017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:55.450017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:55.450017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:55.450017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:55.450017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:55.450017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:55.450017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:55.450017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:55.450017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:55.450017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:09:55.450017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:09:55.450017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:09:55.450017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:55.451034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:55.451966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:55.451966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:55.451966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:09:55.451966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:09:55.451966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:09:55.451966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:09:55.451966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:09:55.451966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:09:55.451966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:09:55.451966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:55.452533 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f319380-96f4-44cd-a5d1-9f71b7ef1c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:55.452533 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f319380-96f4-44cd-a5d1-9f71b7ef1c34 None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:55.452834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f319380-96f4-44cd-a5d1-9f71b7ef1c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:55.453262 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f319380-96f4-44cd-a5d1-9f71b7ef1c34 None admin] PATH_INFO: `/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/6505100494d4496591407c48b48535ee/roles/23ea214daa734346a6da3a0a8c50d23a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:55.455623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 311/615] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:09:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:55.472334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97188792-438e-4a3f-9aae-2ab00185a58b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:55.485235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f319380-96f4-44cd-a5d1-9f71b7ef1c34 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a7317af325f485e8fae49631cb5bc20, user_id=6505100494d4496591407c48b48535ee, role_id=23ea214daa734346a6da3a0a8c50d23a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:55.486960 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f319380-96f4-44cd-a5d1-9f71b7ef1c34 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:55.506270 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 305/616] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:09:55 2022] PUT /identity/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/6505100494d4496591407c48b48535ee/roles/23ea214daa734346a6da3a0a8c50d23a => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:55.664543 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97188792-438e-4a3f-9aae-2ab00185a58b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:55.689471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:55.727358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b24b7-1035-4c6d-9407-f7924aa1458c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:55.728438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b24b7-1035-4c6d-9407-f7924aa1458c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:55.729188 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b24b7-1035-4c6d-9407-f7924aa1458c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:55.729743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b24b7-1035-4c6d-9407-f7924aa1458c None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:55.730906 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b24b7-1035-4c6d-9407-f7924aa1458c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:55.731521 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b24b7-1035-4c6d-9407-f7924aa1458c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:55.733785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b24b7-1035-4c6d-9407-f7924aa1458c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:55.768433 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f319380-96f4-44cd-a5d1-9f71b7ef1c34 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:55.772313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 312/617] 10.222.0.88 () {56 vars in 1135 bytes} [Tue Aug 23 22:09:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:55.914248 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b24b7-1035-4c6d-9407-f7924aa1458c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:55.958046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f319380-96f4-44cd-a5d1-9f71b7ef1c34 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:56.005987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a820afa-d63c-402c-9928-13b5c4a20714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:56.006752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a820afa-d63c-402c-9928-13b5c4a20714 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:56.007132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a820afa-d63c-402c-9928-13b5c4a20714 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:56.007586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a820afa-d63c-402c-9928-13b5c4a20714 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:56.021260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a820afa-d63c-402c-9928-13b5c4a20714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:56.022786 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a820afa-d63c-402c-9928-13b5c4a20714 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:56.025674 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:56.047488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-0a820afa-d63c-402c-9928-13b5c4a20714 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:56.047488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:56.047488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:56.047488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:56.047488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:56.047488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:56.047488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:56.047488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:56.047488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:56.047488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:56.047488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:56.047488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:56.047488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:56.047488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:56.047488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:56.047488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:56.048827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:56.050517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:56.050517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:56.050517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:56.050517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 306/618] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:56.058350 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a820afa-d63c-402c-9928-13b5c4a20714 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:56.228737 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b24b7-1035-4c6d-9407-f7924aa1458c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:56.304552 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f1829-388a-4774-b1ff-fa99aa6cd14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:56.305278 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f1829-388a-4774-b1ff-fa99aa6cd14d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:56.305571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f1829-388a-4774-b1ff-fa99aa6cd14d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:56.305847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f1829-388a-4774-b1ff-fa99aa6cd14d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:56.338820 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f1829-388a-4774-b1ff-fa99aa6cd14d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:56.343780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-5f0f1829-388a-4774-b1ff-fa99aa6cd14d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:56.343780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:56.343780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:56.343780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:56.343780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:56.343780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:56.343780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:56.343780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:56.343780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:56.343780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:56.343780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:56.343780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:56.343780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:56.343780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:56.343780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:56.343780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:56.346140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:56.346140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:56.346140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:56.346140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:56.346140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:56.346140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:56.346140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:56.346140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:56.346140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:56.346140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:56.346140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:56.346140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:56.346140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:56.346140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:56.346140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:56.346140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:56.348447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:56.348447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:56.348447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:56.348447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:56.348447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:56.348447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:56.348447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:56.348447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:56.348447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:56.348447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:56.348447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:56.348447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:56.348447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 313/619] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:09:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 432 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:56.354914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f1829-388a-4774-b1ff-fa99aa6cd14d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:56.397303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a820afa-d63c-402c-9928-13b5c4a20714 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:56.426704 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:56.514162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49627d-0a0e-43d7-96b5-867d0569da0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:56.515275 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49627d-0a0e-43d7-96b5-867d0569da0d None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:56.515897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49627d-0a0e-43d7-96b5-867d0569da0d None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:56.516960 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49627d-0a0e-43d7-96b5-867d0569da0d None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:56.523458 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49627d-0a0e-43d7-96b5-867d0569da0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:56.524106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49627d-0a0e-43d7-96b5-867d0569da0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:56.529522 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49627d-0a0e-43d7-96b5-867d0569da0d None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:56.563774 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 307/620] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 506 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:56.574089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49627d-0a0e-43d7-96b5-867d0569da0d None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:56.619537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:56.646318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f1829-388a-4774-b1ff-fa99aa6cd14d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:56.724916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fe173-c783-470a-99db-0d3e675be3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:56.726070 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fe173-c783-470a-99db-0d3e675be3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:56.726564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fe173-c783-470a-99db-0d3e675be3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:56.727079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fe173-c783-470a-99db-0d3e675be3d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:56.728370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597fe173-c783-470a-99db-0d3e675be3d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:56.728942 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597fe173-c783-470a-99db-0d3e675be3d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:56.731663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597fe173-c783-470a-99db-0d3e675be3d6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:56.769158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 314/621] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:09:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:56.783612 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fe173-c783-470a-99db-0d3e675be3d6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:56.808785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49627d-0a0e-43d7-96b5-867d0569da0d None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:56.887997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d44f6-27f1-499d-b5ca-dd2cbf8f22f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:56.889622 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d44f6-27f1-499d-b5ca-dd2cbf8f22f7 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:56.890374 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d44f6-27f1-499d-b5ca-dd2cbf8f22f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:56.891212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d44f6-27f1-499d-b5ca-dd2cbf8f22f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:56.905120 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d44f6-27f1-499d-b5ca-dd2cbf8f22f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:56.909230 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d44f6-27f1-499d-b5ca-dd2cbf8f22f7 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:56.940652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-8b4d44f6-27f1-499d-b5ca-dd2cbf8f22f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:56.940652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:56.940652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:56.940652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:56.940652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:56.940652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:56.940652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:56.940652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:56.940652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:56.940652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:56.940652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:56.940652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:56.940652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:09:56.940652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:09:56.940652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:56.942449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:56.944280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:09:56.944280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:09:56.944280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:09:56.944280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:09:56.944280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:09:56.944280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:09:56.944280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:09:56.944280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:56.948847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 308/622] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:56.961767 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d44f6-27f1-499d-b5ca-dd2cbf8f22f7 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:57.065656 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:57.074883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fe173-c783-470a-99db-0d3e675be3d6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:57.165144 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac69e8-a240-4a5d-a39e-eac4cb54601d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:57.166152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac69e8-a240-4a5d-a39e-eac4cb54601d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:57.166677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac69e8-a240-4a5d-a39e-eac4cb54601d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:57.167190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac69e8-a240-4a5d-a39e-eac4cb54601d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:57.210051 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac69e8-a240-4a5d-a39e-eac4cb54601d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:57.214655 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac69e8-a240-4a5d-a39e-eac4cb54601d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:57.221481 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:57.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-12ac69e8-a240-4a5d-a39e-eac4cb54601d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:57.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:57.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:57.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:57.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:57.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:57.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:57.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:57.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:57.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:57.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:57.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:57.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:57.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:57.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:57.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:57.279822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:57.280721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:57.280721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:57.280721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:57.280721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 315/623] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 498 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:57.289582 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac69e8-a240-4a5d-a39e-eac4cb54601d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:57.295642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d44f6-27f1-499d-b5ca-dd2cbf8f22f7 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:57.389654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee858ca-83b2-4392-9c2a-c7d9ef0940f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:57.392956 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee858ca-83b2-4392-9c2a-c7d9ef0940f1 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:57.393475 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee858ca-83b2-4392-9c2a-c7d9ef0940f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:57.393960 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee858ca-83b2-4392-9c2a-c7d9ef0940f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:57.395262 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee858ca-83b2-4392-9c2a-c7d9ef0940f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:57.395818 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee858ca-83b2-4392-9c2a-c7d9ef0940f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:57.408519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee858ca-83b2-4392-9c2a-c7d9ef0940f1 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:57.466243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 309/624] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:09:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 506 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:57.474425 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee858ca-83b2-4392-9c2a-c7d9ef0940f1 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:57.508923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac69e8-a240-4a5d-a39e-eac4cb54601d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:57.551244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5941a-bfe9-432b-a478-40bc3782cc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:57.551876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5941a-bfe9-432b-a478-40bc3782cc4f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:57.552156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5941a-bfe9-432b-a478-40bc3782cc4f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:57.552422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5941a-bfe9-432b-a478-40bc3782cc4f None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:57.553249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c5941a-bfe9-432b-a478-40bc3782cc4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:57.553529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c5941a-bfe9-432b-a478-40bc3782cc4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:57.555446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c5941a-bfe9-432b-a478-40bc3782cc4f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:57.571173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 316/625] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:57.572472 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28218c9b-6ecb-42c0-86c3-765eca7c7556 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:57.572746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28218c9b-6ecb-42c0-86c3-765eca7c7556 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:57.573015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28218c9b-6ecb-42c0-86c3-765eca7c7556 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:57.579185 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 317/626] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:57.581305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c3183-1eee-4a03-b4a1-317cd3640b53 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:57.581580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c3183-1eee-4a03-b4a1-317cd3640b53 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:57.581854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c3183-1eee-4a03-b4a1-317cd3640b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:57.589809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:57.604023 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b00c3183-1eee-4a03-b4a1-317cd3640b53 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:57.605265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b00c3183-1eee-4a03-b4a1-317cd3640b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:57.673219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee858ca-83b2-4392-9c2a-c7d9ef0940f1 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:57.711020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 318/627] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:57.712841 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c3183-1eee-4a03-b4a1-317cd3640b53 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:57.729333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e1b55-73bd-4399-b3ff-f47a8ec881bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:57.730496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e1b55-73bd-4399-b3ff-f47a8ec881bb None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:57.731791 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e1b55-73bd-4399-b3ff-f47a8ec881bb None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:57.732247 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e1b55-73bd-4399-b3ff-f47a8ec881bb None admin] PATH_INFO: `/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/601a3defca70428689ee22517d7bcb19/roles/23ea214daa734346a6da3a0a8c50d23a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:57.738376 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:57.762812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e1b55-73bd-4399-b3ff-f47a8ec881bb None admin] RBAC: Authorizing `identity:create_grant(project_id=8a7317af325f485e8fae49631cb5bc20, user_id=601a3defca70428689ee22517d7bcb19, role_id=23ea214daa734346a6da3a0a8c50d23a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:57.770753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e1b55-73bd-4399-b3ff-f47a8ec881bb None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:57.810104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 310/628] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:09:57 2022] PUT /identity/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/601a3defca70428689ee22517d7bcb19/roles/23ea214daa734346a6da3a0a8c50d23a => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:57.919579 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c3183-1eee-4a03-b4a1-317cd3640b53 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:57.979030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c144575-afef-4442-b139-6864af7ab7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:57.979581 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c144575-afef-4442-b139-6864af7ab7aa None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:57.979877 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c144575-afef-4442-b139-6864af7ab7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:57.980117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c144575-afef-4442-b139-6864af7ab7aa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:57.982486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:57.994321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c144575-afef-4442-b139-6864af7ab7aa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:57.995813 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c144575-afef-4442-b139-6864af7ab7aa None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:58.010862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-2c144575-afef-4442-b139-6864af7ab7aa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 22:09:58.010862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:58.010862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:58.010862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:58.010862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:58.010862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:58.010862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:58.010862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:58.010862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:58.010862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:58.010862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:58.010862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:58.010862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:09:58.010862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:09:58.010862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:09:58.010862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:58.012762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:58.014842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:58.014842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:58.014842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:09:58.014842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:09:58.014842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:09:58.014842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:09:58.014842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:09:58.014842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:09:58.014842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 22:09:58.014842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:58.014842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 319/629] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:58.026361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e1b55-73bd-4399-b3ff-f47a8ec881bb None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:58.192068 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e1b55-73bd-4399-b3ff-f47a8ec881bb None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:58.234731 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff9b90-64b9-42cd-a71c-77cb7ad35131 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:58.235647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff9b90-64b9-42cd-a71c-77cb7ad35131 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:58.235890 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff9b90-64b9-42cd-a71c-77cb7ad35131 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:58.241230 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 320/630] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:09:58.242460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c943f4-34f5-4044-a354-af365feaf633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:58.243160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c943f4-34f5-4044-a354-af365feaf633 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:58.243444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c943f4-34f5-4044-a354-af365feaf633 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:58.243602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e65dde-7cb5-4e8d-9b73-0ab489eeca57 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:58.243852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c943f4-34f5-4044-a354-af365feaf633 None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:58.244025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e65dde-7cb5-4e8d-9b73-0ab489eeca57 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:58.244025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e65dde-7cb5-4e8d-9b73-0ab489eeca57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:58.244525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c943f4-34f5-4044-a354-af365feaf633 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:58.244835 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c943f4-34f5-4044-a354-af365feaf633 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:58.246325 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c943f4-34f5-4044-a354-af365feaf633 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:58.276199 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-35e65dde-7cb5-4e8d-9b73-0ab489eeca57 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:58.277474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-35e65dde-7cb5-4e8d-9b73-0ab489eeca57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:58.283409 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 311/631] 10.222.0.88 () {56 vars in 1137 bytes} [Tue Aug 23 22:09:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:58.298323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c943f4-34f5-4044-a354-af365feaf633 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:58.388305 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:58.432457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 321/632] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:58.432457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e65dde-7cb5-4e8d-9b73-0ab489eeca57 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:58.502735 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c943f4-34f5-4044-a354-af365feaf633 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:58.572738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e708a46-9314-4b43-bed7-7e61d51be4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:58.574293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e708a46-9314-4b43-bed7-7e61d51be4cb None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:58.574768 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e708a46-9314-4b43-bed7-7e61d51be4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:58.575230 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e708a46-9314-4b43-bed7-7e61d51be4cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:58.671324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e65dde-7cb5-4e8d-9b73-0ab489eeca57 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:58.714801 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e708a46-9314-4b43-bed7-7e61d51be4cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:58.718980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e708a46-9314-4b43-bed7-7e61d51be4cb None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:58.725252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:58.746143 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be33984-247a-4fce-8f7f-39b3356f16b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:58.746965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be33984-247a-4fce-8f7f-39b3356f16b5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:58.747404 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be33984-247a-4fce-8f7f-39b3356f16b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:58.747853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be33984-247a-4fce-8f7f-39b3356f16b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:58.767090 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be33984-247a-4fce-8f7f-39b3356f16b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:58.768101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-5be33984-247a-4fce-8f7f-39b3356f16b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:58.768101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:58.768101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:58.768101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:58.768101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:58.768101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:58.768101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:58.768101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:58.768101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:58.768101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:58.768101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:58.768101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:58.768101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:58.768101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:58.768101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:58.768101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:58.770571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:58.770571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:58.770571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:58.770571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:58.770571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:58.770571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:58.770571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:58.770571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:58.770571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:58.770571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:58.770571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:58.770571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:58.770571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:58.770571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:58.770571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:58.770571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:58.772063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:58.772063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:58.772063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:58.772063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:58.772063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:58.772063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:58.772063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:58.772063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:58.772063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:58.772063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:58.772063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:58.772063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:58.772063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 322/633] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:09:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:58.772063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f6874-65c3-4376-9925-a401510199da None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:58.774320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f6874-65c3-4376-9925-a401510199da None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:58.774320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f6874-65c3-4376-9925-a401510199da None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:58.774320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 323/634] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:58.774320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f6874-65c3-4376-9925-a401510199da None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:58.820836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:58.898986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 312/635] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:58.901630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a5ce4-1d88-402c-988d-55a5f56a9839 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:58.902328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a5ce4-1d88-402c-988d-55a5f56a9839 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:58.903036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a5ce4-1d88-402c-988d-55a5f56a9839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:58.933542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5b3a5ce4-1d88-402c-988d-55a5f56a9839 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:58.935317 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5b3a5ce4-1d88-402c-988d-55a5f56a9839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:58.978175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f6874-65c3-4376-9925-a401510199da None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:59.022460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3a135-ab59-4736-8b33-46a3f4d9da6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:59.023321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3a135-ab59-4736-8b33-46a3f4d9da6f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:59.023793 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3a135-ab59-4736-8b33-46a3f4d9da6f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:59.024192 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3a135-ab59-4736-8b33-46a3f4d9da6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:59.038251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3a135-ab59-4736-8b33-46a3f4d9da6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:59.043936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-52a3a135-ab59-4736-8b33-46a3f4d9da6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:59.043936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:59.043936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:59.043936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:59.043936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:59.043936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:59.043936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:59.043936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:59.043936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:59.043936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:59.043936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:59.043936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:59.043936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:59.043936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:59.043936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:59.043936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:59.046073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:59.046073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:59.046073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:59.046073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:59.046073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:59.046073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:59.046073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:59.046073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:59.046073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:59.046073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:59.046073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:59.046073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:59.046073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:59.046073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:59.046073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:59.046073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:59.048478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:59.048478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:59.048478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:59.048478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:59.048478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:59.048478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:59.048478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:59.048478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:59.048478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:59.048478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:59.048478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:59.048478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:59.048478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 324/636] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:09:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:59.048478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3a135-ab59-4736-8b33-46a3f4d9da6f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:59.081120 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 313/637] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:59.081120 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a5ce4-1d88-402c-988d-55a5f56a9839 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:59.145893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:59.181386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:59.216049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a5ce4-1d88-402c-988d-55a5f56a9839 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:59.269605 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3a135-ab59-4736-8b33-46a3f4d9da6f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:59.294788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7396fe8-9b51-4d98-925e-e887f4b4f288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:59.296221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7396fe8-9b51-4d98-925e-e887f4b4f288 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:59.297022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7396fe8-9b51-4d98-925e-e887f4b4f288 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:59.299242 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7396fe8-9b51-4d98-925e-e887f4b4f288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:59.320114 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7396fe8-9b51-4d98-925e-e887f4b4f288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:59.323314 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7396fe8-9b51-4d98-925e-e887f4b4f288 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:59.341915 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1691b67-e3cc-48b4-81bd-2ad6d7b08194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:59.342895 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1691b67-e3cc-48b4-81bd-2ad6d7b08194 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:59.343393 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1691b67-e3cc-48b4-81bd-2ad6d7b08194 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:59.343884 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1691b67-e3cc-48b4-81bd-2ad6d7b08194 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:59.345979 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1691b67-e3cc-48b4-81bd-2ad6d7b08194 None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:59.346588 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1691b67-e3cc-48b4-81bd-2ad6d7b08194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:59.349156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1691b67-e3cc-48b4-81bd-2ad6d7b08194 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:59.356929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-b7396fe8-9b51-4d98-925e-e887f4b4f288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:59.356929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:59.356929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:59.356929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:59.356929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:59.356929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:59.356929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:59.356929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:59.356929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:59.356929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:59.356929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:59.356929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:59.356929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:59.356929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:59.356929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:59.356929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:59.359007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:59.362167 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:59.362167 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:59.362167 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:59.371973 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 325/638] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:09:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:59.371973 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1691b67-e3cc-48b4-81bd-2ad6d7b08194 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:59.373046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 314/639] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:59.375006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7396fe8-9b51-4d98-925e-e887f4b4f288 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:59.558877 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1691b67-e3cc-48b4-81bd-2ad6d7b08194 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:59.563942 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7396fe8-9b51-4d98-925e-e887f4b4f288 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:59.574229 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:59.616583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:59.631501 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bd19f-0c36-4fa4-ab66-63595e482ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:59.631889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9804e-453e-41e5-abf0-9978bde4b3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:59.632874 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9804e-453e-41e5-abf0-9978bde4b3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:59.633388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9804e-453e-41e5-abf0-9978bde4b3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:59.633810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9804e-453e-41e5-abf0-9978bde4b3c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:59.634185 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bd19f-0c36-4fa4-ab66-63595e482ead None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:59.634945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bd19f-0c36-4fa4-ab66-63595e482ead None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:59.635106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce9804e-453e-41e5-abf0-9978bde4b3c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:59.635537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce9804e-453e-41e5-abf0-9978bde4b3c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:59.636015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bd19f-0c36-4fa4-ab66-63595e482ead None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:59.637614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285bd19f-0c36-4fa4-ab66-63595e482ead None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:59.638271 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce9804e-453e-41e5-abf0-9978bde4b3c2 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:59.638691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285bd19f-0c36-4fa4-ab66-63595e482ead None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:59.642156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285bd19f-0c36-4fa4-ab66-63595e482ead None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:59.658734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 326/640] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:09:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:59.660643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bd19f-0c36-4fa4-ab66-63595e482ead None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:59.669914 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 315/641] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:59.671872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9804e-453e-41e5-abf0-9978bde4b3c2 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:59.830027 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bd19f-0c36-4fa4-ab66-63595e482ead None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:59.873928 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9804e-453e-41e5-abf0-9978bde4b3c2 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:59.877437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3223c-fbed-481c-a4c6-71e352ee1960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:59.878553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3223c-fbed-481c-a4c6-71e352ee1960 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:59.879056 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3223c-fbed-481c-a4c6-71e352ee1960 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:59.879563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3223c-fbed-481c-a4c6-71e352ee1960 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:59.900011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3223c-fbed-481c-a4c6-71e352ee1960 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:59.902871 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3223c-fbed-481c-a4c6-71e352ee1960 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:59.928680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-74d3223c-fbed-481c-a4c6-71e352ee1960 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:59.928680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:59.928680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:59.928680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:59.928680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:59.928680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:59.928680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:59.928680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:59.928680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:59.928680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:59.928680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:59.928680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:59.928680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:09:59.928680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:09:59.928680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:09:59.928680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:59.930522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:09:59.932281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:09:59.932281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:09:59.932281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:09:59.933450 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 327/642] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:59.934452 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3223c-fbed-481c-a4c6-71e352ee1960 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:59.947089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c85e4d-cd9b-4c55-8f57-6e753156fbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:59.947089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c85e4d-cd9b-4c55-8f57-6e753156fbfa None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:59.947089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c85e4d-cd9b-4c55-8f57-6e753156fbfa None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:59.947089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c85e4d-cd9b-4c55-8f57-6e753156fbfa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:59.967729 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c85e4d-cd9b-4c55-8f57-6e753156fbfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:59.974303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-20c85e4d-cd9b-4c55-8f57-6e753156fbfa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:59.974303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:59.974303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:59.974303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:59.974303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:59.974303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:59.974303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:59.974303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:59.974303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:59.974303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:59.974303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:59.974303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:59.974303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:59.974303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:59.974303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:59.974303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:59.975398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:59.975398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:59.975398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:59.975398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:59.975398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:59.975398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:59.975398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:59.975398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:59.975398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:59.975398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:59.975398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:59.975398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:59.975398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:59.975398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:59.975398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:59.975398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:59.976268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:59.976268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:59.976268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:59.976268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:59.976268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:59.976268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:59.976268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:59.976268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:59.976268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:59.976268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:59.976268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:09:59.976268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:09:59.978874 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 316/643] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:09:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:59.979954 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c85e4d-cd9b-4c55-8f57-6e753156fbfa None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:59.989869 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:00.073378 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:00.132648 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3223c-fbed-481c-a4c6-71e352ee1960 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:00.196508 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf74804-2c7a-4705-b71c-78db9660e4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:00.197534 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf74804-2c7a-4705-b71c-78db9660e4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:00.198154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf74804-2c7a-4705-b71c-78db9660e4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:00.198592 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf74804-2c7a-4705-b71c-78db9660e4b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:00.207018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c85e4d-cd9b-4c55-8f57-6e753156fbfa None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:00.216697 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf74804-2c7a-4705-b71c-78db9660e4b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:00.218202 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf74804-2c7a-4705-b71c-78db9660e4b5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:00.239369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-cdf74804-2c7a-4705-b71c-78db9660e4b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:00.239369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:00.239369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:00.239369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:00.239369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:00.239369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:00.239369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:00.239369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:00.239369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:00.239369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:00.239369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:00.239369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:00.239369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:00.239369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:00.239369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:00.239369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:00.245769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:00.247102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:00.247102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:00.247102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:00.247102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 328/644] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:00.247102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf74804-2c7a-4705-b71c-78db9660e4b5 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:00.272606 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddad31e-3b1a-4b5c-850e-f3d4038e70e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:00.273899 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddad31e-3b1a-4b5c-850e-f3d4038e70e8 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:00.274732 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddad31e-3b1a-4b5c-850e-f3d4038e70e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:00.274984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddad31e-3b1a-4b5c-850e-f3d4038e70e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:00.290177 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddad31e-3b1a-4b5c-850e-f3d4038e70e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:00.293083 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddad31e-3b1a-4b5c-850e-f3d4038e70e8 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:00.316950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-4ddad31e-3b1a-4b5c-850e-f3d4038e70e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:00.316950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:00.316950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:00.316950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:00.316950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:00.316950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:00.316950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:00.316950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:00.316950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:00.316950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:00.316950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:00.316950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:00.316950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:00.316950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:00.316950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:00.318019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:00.318973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:00.318973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:00.318973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:00.318973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:00.318973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:00.318973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:00.318973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:00.318973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:00.319821 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 317/645] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:00.321107 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddad31e-3b1a-4b5c-850e-f3d4038e70e8 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:00.358063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:00.416215 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf74804-2c7a-4705-b71c-78db9660e4b5 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:00.457122 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49e570-70d7-4565-8781-7e5d16e45aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:00.457746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e570-70d7-4565-8781-7e5d16e45aed None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:00.458027 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e570-70d7-4565-8781-7e5d16e45aed None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:00.458302 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e570-70d7-4565-8781-7e5d16e45aed None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:00.458968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49e570-70d7-4565-8781-7e5d16e45aed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:00.459269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49e570-70d7-4565-8781-7e5d16e45aed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:00.460675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49e570-70d7-4565-8781-7e5d16e45aed None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:00.479316 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 329/646] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:00.479316 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49e570-70d7-4565-8781-7e5d16e45aed None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:00.486154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddad31e-3b1a-4b5c-850e-f3d4038e70e8 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:00.496229 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:00.537330 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3da96-1eeb-485f-bb7b-ce3a55d267b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:00.538353 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3da96-1eeb-485f-bb7b-ce3a55d267b0 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:00.538844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3da96-1eeb-485f-bb7b-ce3a55d267b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:00.539317 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3da96-1eeb-485f-bb7b-ce3a55d267b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:00.540486 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3da96-1eeb-485f-bb7b-ce3a55d267b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:00.540833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3da96-1eeb-485f-bb7b-ce3a55d267b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:00.542811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3da96-1eeb-485f-bb7b-ce3a55d267b0 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:00.554038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 318/647] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:00.554904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3da96-1eeb-485f-bb7b-ce3a55d267b0 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:00.645737 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49e570-70d7-4565-8781-7e5d16e45aed None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:00.684089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:00.687550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3da96-1eeb-485f-bb7b-ce3a55d267b0 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:00.692527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a01f88-0e20-42ab-9c0e-3f2d2bdbb4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:00.693295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a01f88-0e20-42ab-9c0e-3f2d2bdbb4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:00.693695 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a01f88-0e20-42ab-9c0e-3f2d2bdbb4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:00.694051 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a01f88-0e20-42ab-9c0e-3f2d2bdbb4f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:00.694782 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a01f88-0e20-42ab-9c0e-3f2d2bdbb4f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:00.695148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a01f88-0e20-42ab-9c0e-3f2d2bdbb4f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:00.696638 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a01f88-0e20-42ab-9c0e-3f2d2bdbb4f2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:00.711582 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 330/648] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:00.713157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97a5ba-8036-4f0d-a5af-b632eba26074 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:00.713459 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97a5ba-8036-4f0d-a5af-b632eba26074 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:00.713742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97a5ba-8036-4f0d-a5af-b632eba26074 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:00.719968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 331/649] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:00.719968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97a5ba-8036-4f0d-a5af-b632eba26074 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:00.747282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b118a4-ca07-4af4-8309-c3de0a9c1d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:00.748368 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b118a4-ca07-4af4-8309-c3de0a9c1d0e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:00.748826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b118a4-ca07-4af4-8309-c3de0a9c1d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:00.749349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b118a4-ca07-4af4-8309-c3de0a9c1d0e None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:00.750969 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b118a4-ca07-4af4-8309-c3de0a9c1d0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:00.751526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b118a4-ca07-4af4-8309-c3de0a9c1d0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:00.754227 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b118a4-ca07-4af4-8309-c3de0a9c1d0e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:00.771239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 319/650] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:00.772743 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b118a4-ca07-4af4-8309-c3de0a9c1d0e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:00.821932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:00.854400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97a5ba-8036-4f0d-a5af-b632eba26074 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:00.896157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24b78c-aa0d-4388-ae12-e0ce43f61487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:00.896938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24b78c-aa0d-4388-ae12-e0ce43f61487 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:00.897510 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24b78c-aa0d-4388-ae12-e0ce43f61487 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:00.897946 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24b78c-aa0d-4388-ae12-e0ce43f61487 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:00.908438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24b78c-aa0d-4388-ae12-e0ce43f61487 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:00.910239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24b78c-aa0d-4388-ae12-e0ce43f61487 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:00.916075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b118a4-ca07-4af4-8309-c3de0a9c1d0e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:00.925747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-3b24b78c-aa0d-4388-ae12-e0ce43f61487 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:00.925747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:00.925747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:00.925747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:00.925747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:00.925747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:00.925747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:00.925747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:00.925747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:00.925747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:00.925747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:00.925747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:00.925747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:00.925747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:00.925747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:00.925747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:00.926989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:00.928491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:00.928491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:00.928491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:00.928491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:00.928491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:00.928491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:00.928491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:00.928491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:00.928491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:00.928491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:00.928491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 332/651] 10.222.0.88 () {56 vars in 1093 bytes} [Tue Aug 23 22:10:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:00.928491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24b78c-aa0d-4388-ae12-e0ce43f61487 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:00.963046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244de7ac-c688-42c4-85c9-18b33a416f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:00.963570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244de7ac-c688-42c4-85c9-18b33a416f59 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:00.963847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244de7ac-c688-42c4-85c9-18b33a416f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:00.964124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244de7ac-c688-42c4-85c9-18b33a416f59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:00.976400 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244de7ac-c688-42c4-85c9-18b33a416f59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:00.977886 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244de7ac-c688-42c4-85c9-18b33a416f59 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:00.994829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-244de7ac-c688-42c4-85c9-18b33a416f59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:00.994829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:00.994829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:00.994829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:00.994829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:00.994829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:00.994829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:00.994829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:00.994829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:00.994829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:00.994829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:00.994829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:00.994829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:00.994829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:00.994829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:00.994829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:00.996895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:00.998689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:00.998689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:00.998689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:00.998689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 320/652] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:00.999124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a856ac5-dc46-4e51-a075-5b3195265493 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:00.999274 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a856ac5-dc46-4e51-a075-5b3195265493 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:00.999747 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a856ac5-dc46-4e51-a075-5b3195265493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:01.021568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-6a856ac5-dc46-4e51-a075-5b3195265493 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:01.022805 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6a856ac5-dc46-4e51-a075-5b3195265493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:01.034825 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:01.099195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24b78c-aa0d-4388-ae12-e0ce43f61487 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:01.120020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 321/653] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:01.121081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a856ac5-dc46-4e51-a075-5b3195265493 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:01.141446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:01.151616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f225b94-4c78-4817-acba-36151074c174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:01.152289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f225b94-4c78-4817-acba-36151074c174 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:01.152572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f225b94-4c78-4817-acba-36151074c174 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:01.153282 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f225b94-4c78-4817-acba-36151074c174 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:01.164612 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f225b94-4c78-4817-acba-36151074c174 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:01.166752 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f225b94-4c78-4817-acba-36151074c174 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:01.192527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-4f225b94-4c78-4817-acba-36151074c174 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 22:10:01.192527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:01.192527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:01.192527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:01.192527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:01.192527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:01.192527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:01.192527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:01.192527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:01.192527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:01.192527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:01.192527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:01.192527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:01.192527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:01.192527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:01.192527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:01.193714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:01.194644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:01.194644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:01.194644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:01.194644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:01.194644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:01.194644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:01.194644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:01.194644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:01.194644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 22:10:01.194644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:01.194644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 333/654] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:01.194644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f225b94-4c78-4817-acba-36151074c174 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:01.291694 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a856ac5-dc46-4e51-a075-5b3195265493 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:01.357815 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf3feb-ad7b-4b5e-b362-a0db6e69d711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:01.358697 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf3feb-ad7b-4b5e-b362-a0db6e69d711 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:01.359440 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf3feb-ad7b-4b5e-b362-a0db6e69d711 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:01.359440 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf3feb-ad7b-4b5e-b362-a0db6e69d711 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:01.360962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf3feb-ad7b-4b5e-b362-a0db6e69d711 None admin] RBAC: Adding query filter params (role.id=23ea214daa734346a6da3a0a8c50d23a, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=7d9ebd88431d46348089bc195491fca5) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:01.361567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf3feb-ad7b-4b5e-b362-a0db6e69d711 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:01.364169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf3feb-ad7b-4b5e-b362-a0db6e69d711 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:01.376611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 322/655] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:01 2022] GET /identity/v3/role_assignments?user.id=7d9ebd88431d46348089bc195491fca5&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=23ea214daa734346a6da3a0a8c50d23a => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:01.378145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf3feb-ad7b-4b5e-b362-a0db6e69d711 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:01.397430 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f225b94-4c78-4817-acba-36151074c174 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:01.437010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:01.474132 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17880778-2810-463d-8163-4af69edf6516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:01.475699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17880778-2810-463d-8163-4af69edf6516 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:01.476233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17880778-2810-463d-8163-4af69edf6516 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:01.476698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17880778-2810-463d-8163-4af69edf6516 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:01.477975 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17880778-2810-463d-8163-4af69edf6516 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:01.478734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17880778-2810-463d-8163-4af69edf6516 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:01.480977 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17880778-2810-463d-8163-4af69edf6516 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:01.505442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 334/656] 10.222.0.88 () {56 vars in 1133 bytes} [Tue Aug 23 22:10:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:01.506749 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17880778-2810-463d-8163-4af69edf6516 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:01.559904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:01.611454 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf3feb-ad7b-4b5e-b362-a0db6e69d711 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:01.667796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31d255-7b22-475a-81a7-460d49c24b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:01.668892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31d255-7b22-475a-81a7-460d49c24b20 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:01.669459 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31d255-7b22-475a-81a7-460d49c24b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:01.669974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31d255-7b22-475a-81a7-460d49c24b20 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:01.671223 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31d255-7b22-475a-81a7-460d49c24b20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:01.671738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31d255-7b22-475a-81a7-460d49c24b20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:01.674495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31d255-7b22-475a-81a7-460d49c24b20 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:01.694100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 323/657] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:01.694830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31d255-7b22-475a-81a7-460d49c24b20 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:01.706857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17880778-2810-463d-8163-4af69edf6516 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:01.758825 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d1ee1-ea69-42fd-827b-ef00bcb7c50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:01.759854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d1ee1-ea69-42fd-827b-ef00bcb7c50f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:01.760338 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d1ee1-ea69-42fd-827b-ef00bcb7c50f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:01.760866 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d1ee1-ea69-42fd-827b-ef00bcb7c50f None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:01.767732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d1ee1-ea69-42fd-827b-ef00bcb7c50f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:01.768280 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d1ee1-ea69-42fd-827b-ef00bcb7c50f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:01.771058 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d1ee1-ea69-42fd-827b-ef00bcb7c50f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:01.805928 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 335/658] 10.222.0.88 () {56 vars in 1139 bytes} [Tue Aug 23 22:10:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:01.806932 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d1ee1-ea69-42fd-827b-ef00bcb7c50f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:01.858296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:01.879434 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31d255-7b22-475a-81a7-460d49c24b20 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:01.947658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b12cb4-4149-4ce1-b058-9e91d39eb677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:01.949095 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b12cb4-4149-4ce1-b058-9e91d39eb677 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:01.949791 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b12cb4-4149-4ce1-b058-9e91d39eb677 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:01.950518 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b12cb4-4149-4ce1-b058-9e91d39eb677 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:01.954940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:01.960062 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d1ee1-ea69-42fd-827b-ef00bcb7c50f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:01.970877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b12cb4-4149-4ce1-b058-9e91d39eb677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:01.980502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-94b12cb4-4149-4ce1-b058-9e91d39eb677 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:01.980502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:01.980502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:01.980502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:01.980502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:01.980502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:01.980502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:01.980502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:01.980502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:01.980502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:01.980502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:01.980502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:01.980502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:01.980502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:01.980502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:01.980502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:01.982439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:01.982439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:01.982439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:01.982439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:01.982439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:01.982439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:01.982439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:01.982439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:01.982439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:01.982439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:01.982439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:01.982439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:01.982439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:01.982439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:01.982439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:01.982439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:01.984170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:01.984170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:01.984170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:01.984170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:01.984170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:01.984170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:01.984170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:01.984170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:01.984170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:01.984170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:01.984170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:01.984170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:01.984170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 324/659] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:01.990144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b12cb4-4149-4ce1-b058-9e91d39eb677 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:02.016404 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03a58b-e425-4565-a181-a18833b6c9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:02.017318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03a58b-e425-4565-a181-a18833b6c9ae None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:02.017862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03a58b-e425-4565-a181-a18833b6c9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:02.017862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03a58b-e425-4565-a181-a18833b6c9ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:02.122668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf03a58b-e425-4565-a181-a18833b6c9ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:02.124048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf03a58b-e425-4565-a181-a18833b6c9ae None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:02.144676 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b12cb4-4149-4ce1-b058-9e91d39eb677 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:02.164852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:02.207558 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924251be-73b1-44c3-87eb-0564c3089f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:02.208959 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924251be-73b1-44c3-87eb-0564c3089f6b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:02.209639 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924251be-73b1-44c3-87eb-0564c3089f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:02.210348 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924251be-73b1-44c3-87eb-0564c3089f6b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:02.224573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924251be-73b1-44c3-87eb-0564c3089f6b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:02.227736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924251be-73b1-44c3-87eb-0564c3089f6b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:02.252109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-924251be-73b1-44c3-87eb-0564c3089f6b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 22:10:02.252109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:02.252109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:02.252109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:02.252109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:02.252109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:02.252109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:02.252109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:02.252109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:02.252109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:02.252109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:02.252109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:02.252109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:02.252109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:02.252109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:02.252109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:02.254104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:02.255989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:02.255989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:02.255989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:02.255989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:02.255989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:02.255989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:02.255989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:02.255989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:02.255989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 22:10:02.255989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:02.261248 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 325/660] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:02.262950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924251be-73b1-44c3-87eb-0564c3089f6b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:02.317044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 336/661] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:02.318994 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03a58b-e425-4565-a181-a18833b6c9ae None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:02.350641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:02.435436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924251be-73b1-44c3-87eb-0564c3089f6b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:02.465911 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03a58b-e425-4565-a181-a18833b6c9ae None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:02.495879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ddc23-6808-4f78-b5d8-3a06aa2d4123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:02.496922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ddc23-6808-4f78-b5d8-3a06aa2d4123 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:02.497453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ddc23-6808-4f78-b5d8-3a06aa2d4123 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:02.497956 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ddc23-6808-4f78-b5d8-3a06aa2d4123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:02.509922 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:02.524461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e50c7-0428-432d-9676-02a0cbae4b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:02.525551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e50c7-0428-432d-9676-02a0cbae4b26 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:02.526054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e50c7-0428-432d-9676-02a0cbae4b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:02.526551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e50c7-0428-432d-9676-02a0cbae4b26 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:02.527786 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e50c7-0428-432d-9676-02a0cbae4b26 None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:02.528331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e50c7-0428-432d-9676-02a0cbae4b26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:02.531041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e50c7-0428-432d-9676-02a0cbae4b26 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:02.546636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 337/662] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:02.548254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e50c7-0428-432d-9676-02a0cbae4b26 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:02.611030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ddc23-6808-4f78-b5d8-3a06aa2d4123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:02.613679 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ddc23-6808-4f78-b5d8-3a06aa2d4123 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:02.713396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:02.730762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e50c7-0428-432d-9676-02a0cbae4b26 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:02.780488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d3c47-ea7c-40b3-84c0-f013aac11286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:02.781187 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d3c47-ea7c-40b3-84c0-f013aac11286 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:02.782207 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d3c47-ea7c-40b3-84c0-f013aac11286 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:02.782489 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d3c47-ea7c-40b3-84c0-f013aac11286 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:02.788101 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 326/663] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:02.788385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d3c47-ea7c-40b3-84c0-f013aac11286 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:02.788385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d3c47-ea7c-40b3-84c0-f013aac11286 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:02.788547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ddc23-6808-4f78-b5d8-3a06aa2d4123 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:02.788632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d3c47-ea7c-40b3-84c0-f013aac11286 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:02.815477 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 338/664] 10.222.0.88 () {56 vars in 1137 bytes} [Tue Aug 23 22:10:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:02.821359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d3c47-ea7c-40b3-84c0-f013aac11286 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:02.900923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:02.940525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ddc23-6808-4f78-b5d8-3a06aa2d4123 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:02.967775 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d3c47-ea7c-40b3-84c0-f013aac11286 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:03.001537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491ec24-2efe-4c7e-aee7-450ffaca914d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:03.002346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491ec24-2efe-4c7e-aee7-450ffaca914d None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:03.002626 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491ec24-2efe-4c7e-aee7-450ffaca914d None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:03.002967 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491ec24-2efe-4c7e-aee7-450ffaca914d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:03.014245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926279a-e8e1-4f47-9dbd-b3eae64f0ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:03.015071 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926279a-e8e1-4f47-9dbd-b3eae64f0ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:03.015547 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926279a-e8e1-4f47-9dbd-b3eae64f0ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:03.016662 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926279a-e8e1-4f47-9dbd-b3eae64f0ca4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:03.020733 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491ec24-2efe-4c7e-aee7-450ffaca914d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:03.023328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491ec24-2efe-4c7e-aee7-450ffaca914d None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:03.030030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926279a-e8e1-4f47-9dbd-b3eae64f0ca4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:03.032192 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926279a-e8e1-4f47-9dbd-b3eae64f0ca4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:03.050349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-9491ec24-2efe-4c7e-aee7-450ffaca914d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:03.050349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:03.050349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:03.050349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:03.050349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:03.050349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:03.050349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:03.050349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:03.050349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:03.050349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:03.050349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:03.050349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:03.050349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:03.050349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:03.050349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:03.050349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:03.052171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:03.059081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:03.059081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:03.059081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:03.059081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 327/665] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:03.059081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491ec24-2efe-4c7e-aee7-450ffaca914d None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:03.059376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-0926279a-e8e1-4f47-9dbd-b3eae64f0ca4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:03.059376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:03.059376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:03.059376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:03.059376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:03.059376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:03.059376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:03.059376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:03.059376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:03.059376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:03.059376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:03.059376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:03.059376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:03.059376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:03.059376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:03.059376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:03.060303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:03.065566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:03.065566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:03.065566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:03.065566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 339/666] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:03.065566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926279a-e8e1-4f47-9dbd-b3eae64f0ca4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:03.077326 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:03.224623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491ec24-2efe-4c7e-aee7-450ffaca914d None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:03.245738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:03.251947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926279a-e8e1-4f47-9dbd-b3eae64f0ca4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:03.279581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aac02c-2a40-4c39-b82c-96500473b4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:03.280381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aac02c-2a40-4c39-b82c-96500473b4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:03.280846 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aac02c-2a40-4c39-b82c-96500473b4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:03.281004 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aac02c-2a40-4c39-b82c-96500473b4f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:03.296121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aac02c-2a40-4c39-b82c-96500473b4f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:03.299047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aac02c-2a40-4c39-b82c-96500473b4f9 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:03.313810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa1c54-c72d-4003-a581-877586c4728b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:03.314765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa1c54-c72d-4003-a581-877586c4728b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:03.315193 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa1c54-c72d-4003-a581-877586c4728b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:03.315606 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa1c54-c72d-4003-a581-877586c4728b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:03.320209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-36aac02c-2a40-4c39-b82c-96500473b4f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:03.320209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:03.320209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:03.320209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:03.320209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:03.320209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:03.320209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:03.320209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:03.320209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:03.320209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:03.320209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:03.320209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:03.320209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:03.320209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:03.320209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:03.320209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:03.322092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:03.323324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:03.323324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:03.323324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:03.326592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 328/667] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:03.327717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aac02c-2a40-4c39-b82c-96500473b4f9 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:03.435566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:03.456626 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa1c54-c72d-4003-a581-877586c4728b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:03.459240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa1c54-c72d-4003-a581-877586c4728b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:03.517370 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aac02c-2a40-4c39-b82c-96500473b4f9 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:03.572551 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e7570-ac0a-42ce-817f-242fd8687345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:03.573397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e7570-ac0a-42ce-817f-242fd8687345 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:03.573783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e7570-ac0a-42ce-817f-242fd8687345 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:03.574147 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e7570-ac0a-42ce-817f-242fd8687345 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:03.574897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e7570-ac0a-42ce-817f-242fd8687345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:03.575291 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e7570-ac0a-42ce-817f-242fd8687345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:03.576794 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e7570-ac0a-42ce-817f-242fd8687345 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:03.594686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 329/668] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:03.596204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e7570-ac0a-42ce-817f-242fd8687345 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:03.610243 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:03.617241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 340/669] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:03.618154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa1c54-c72d-4003-a581-877586c4728b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:03.725935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e7570-ac0a-42ce-817f-242fd8687345 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:03.745307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa1c54-c72d-4003-a581-877586c4728b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:03.751139 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:03.778002 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194c9aa-9d56-4ec0-a6d1-c5e0f4929114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:03.778604 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194c9aa-9d56-4ec0-a6d1-c5e0f4929114 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:03.779015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194c9aa-9d56-4ec0-a6d1-c5e0f4929114 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:03.779424 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194c9aa-9d56-4ec0-a6d1-c5e0f4929114 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:03.780216 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194c9aa-9d56-4ec0-a6d1-c5e0f4929114 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:03.780652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194c9aa-9d56-4ec0-a6d1-c5e0f4929114 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:03.783038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194c9aa-9d56-4ec0-a6d1-c5e0f4929114 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:03.794036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ca7a6-82ec-4f33-af65-092c2e0115ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:03.795021 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ca7a6-82ec-4f33-af65-092c2e0115ae None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:03.795363 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ca7a6-82ec-4f33-af65-092c2e0115ae None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:03.795816 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ca7a6-82ec-4f33-af65-092c2e0115ae None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:03.796984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ca7a6-82ec-4f33-af65-092c2e0115ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:03.797533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ca7a6-82ec-4f33-af65-092c2e0115ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:03.799811 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ca7a6-82ec-4f33-af65-092c2e0115ae None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:03.804983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 330/670] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:03.806644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b6dfb-3c5d-40a9-9788-a07f39b16eac None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:03.807133 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b6dfb-3c5d-40a9-9788-a07f39b16eac None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:03.807615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b6dfb-3c5d-40a9-9788-a07f39b16eac None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:03.810328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 331/671] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:03.811450 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b6dfb-3c5d-40a9-9788-a07f39b16eac None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:03.819175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 341/672] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:03.819432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ca7a6-82ec-4f33-af65-092c2e0115ae None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:03.916341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:03.960952 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b6dfb-3c5d-40a9-9788-a07f39b16eac None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:03.976741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ca7a6-82ec-4f33-af65-092c2e0115ae None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:04.012143 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55fefa-ebf2-4af6-b27a-4076384c8070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:04.013208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55fefa-ebf2-4af6-b27a-4076384c8070 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.013667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55fefa-ebf2-4af6-b27a-4076384c8070 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.014113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55fefa-ebf2-4af6-b27a-4076384c8070 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.021628 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d916a8-b090-42b1-9963-7f8e753e3417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:04.022655 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d916a8-b090-42b1-9963-7f8e753e3417 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.023158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d916a8-b090-42b1-9963-7f8e753e3417 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.023655 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d916a8-b090-42b1-9963-7f8e753e3417 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.027018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55fefa-ebf2-4af6-b27a-4076384c8070 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:04.029859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55fefa-ebf2-4af6-b27a-4076384c8070 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:04.043342 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d916a8-b090-42b1-9963-7f8e753e3417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:04.045974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d916a8-b090-42b1-9963-7f8e753e3417 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:04.052476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-ea55fefa-ebf2-4af6-b27a-4076384c8070 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:04.052476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:04.052476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:04.052476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:04.052476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:04.052476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:04.052476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:04.052476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:04.052476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:04.052476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:04.052476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:04.052476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:04.052476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:04.052476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:04.052476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:04.053438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:04.054406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:04.054406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:04.054406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:04.054406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:04.054406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:04.054406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:04.054406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:04.054406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:04.055125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 332/673] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:04.056391 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55fefa-ebf2-4af6-b27a-4076384c8070 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:04.072500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-32d916a8-b090-42b1-9963-7f8e753e3417 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:04.072500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:04.072500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:04.072500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:04.072500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:04.072500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:04.072500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:04.072500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:04.072500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:04.072500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:04.072500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:04.072500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:04.072500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:04.072500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:04.072500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:04.072500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:04.073554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:04.074436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:04.074436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:04.074436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:04.074436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 342/674] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:04.076428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd2b65-6b36-4d2c-ad40-f33062167a5d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.076763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd2b65-6b36-4d2c-ad40-f33062167a5d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.077157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd2b65-6b36-4d2c-ad40-f33062167a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.097389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-54fd2b65-6b36-4d2c-ad40-f33062167a5d None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:04.098619 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-54fd2b65-6b36-4d2c-ad40-f33062167a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:04.137682 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:04.204770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 343/675] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:04.204770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd2b65-6b36-4d2c-ad40-f33062167a5d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:04.252128 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55fefa-ebf2-4af6-b27a-4076384c8070 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:04.279075 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:04.315743 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bccf9-f040-4697-987d-775491b5e2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:04.316614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bccf9-f040-4697-987d-775491b5e2cd None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.317038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bccf9-f040-4697-987d-775491b5e2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.317346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bccf9-f040-4697-987d-775491b5e2cd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.328834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bccf9-f040-4697-987d-775491b5e2cd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:04.330547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bccf9-f040-4697-987d-775491b5e2cd None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:04.346836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-689bccf9-f040-4697-987d-775491b5e2cd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:04.346836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:04.346836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:04.346836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:04.346836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:04.346836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:04.346836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:04.346836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:04.346836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:04.346836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:04.346836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:04.346836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:04.346836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:04.346836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:04.346836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:04.346836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:04.348702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:04.350781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:04.350781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:04.350781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:04.350781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:04.350781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:04.350781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:04.350781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:04.350781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:04.350781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:04.350781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:04.350781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 333/676] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:04.350781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bccf9-f040-4697-987d-775491b5e2cd None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:04.377403 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd2b65-6b36-4d2c-ad40-f33062167a5d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:04.424838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d49b8-5c74-427a-afc9-6950f288ef22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:04.425614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d49b8-5c74-427a-afc9-6950f288ef22 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.425899 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d49b8-5c74-427a-afc9-6950f288ef22 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.426169 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d49b8-5c74-427a-afc9-6950f288ef22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.434192 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d49b8-5c74-427a-afc9-6950f288ef22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:04.435701 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d49b8-5c74-427a-afc9-6950f288ef22 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:04.449310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-bd4d49b8-5c74-427a-afc9-6950f288ef22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:04.449310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:04.449310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:04.449310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:04.449310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:04.449310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:04.449310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:04.449310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:04.449310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:04.449310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:04.449310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:04.449310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:04.449310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:04.449310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:04.449310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:04.450286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:04.451195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:04.451195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:04.451195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:04.451195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:04.451195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:04.451195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:04.451195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:04.451195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:04.451195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 344/677] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:04.451686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d49b8-5c74-427a-afc9-6950f288ef22 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:04.531810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:04.538412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bccf9-f040-4697-987d-775491b5e2cd None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:04.591960 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5b92a-250a-42f8-818d-fbac2c4335c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:04.592505 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5b92a-250a-42f8-818d-fbac2c4335c2 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.592781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5b92a-250a-42f8-818d-fbac2c4335c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.593084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5b92a-250a-42f8-818d-fbac2c4335c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.593833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5b92a-250a-42f8-818d-fbac2c4335c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:04.594630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5b92a-250a-42f8-818d-fbac2c4335c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:04.596161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5b92a-250a-42f8-818d-fbac2c4335c2 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:04.613906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 334/678] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:04.615390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5b92a-250a-42f8-818d-fbac2c4335c2 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:04.632814 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d49b8-5c74-427a-afc9-6950f288ef22 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:04.634801 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:04.683003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651d022-3d40-4391-8edd-fd63f75a407b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:04.683632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651d022-3d40-4391-8edd-fd63f75a407b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.683917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651d022-3d40-4391-8edd-fd63f75a407b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.684198 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651d022-3d40-4391-8edd-fd63f75a407b None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.684862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651d022-3d40-4391-8edd-fd63f75a407b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:04.685196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651d022-3d40-4391-8edd-fd63f75a407b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:04.686633 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651d022-3d40-4391-8edd-fd63f75a407b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:04.700420 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 345/679] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:04.701329 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651d022-3d40-4391-8edd-fd63f75a407b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:04.771046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5b92a-250a-42f8-818d-fbac2c4335c2 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:04.820077 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c1af5-40d5-4b66-bbad-663c8dc01ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:04.820712 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c1af5-40d5-4b66-bbad-663c8dc01ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.820987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c1af5-40d5-4b66-bbad-663c8dc01ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.821313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c1af5-40d5-4b66-bbad-663c8dc01ea3 None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.822081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c1af5-40d5-4b66-bbad-663c8dc01ea3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:04.822382 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c1af5-40d5-4b66-bbad-663c8dc01ea3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:04.823853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c1af5-40d5-4b66-bbad-663c8dc01ea3 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:04.845573 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651d022-3d40-4391-8edd-fd63f75a407b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:04.848395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 335/680] 10.222.0.88 () {56 vars in 1135 bytes} [Tue Aug 23 22:10:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:04.849609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c1af5-40d5-4b66-bbad-663c8dc01ea3 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:04.873361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:04.898608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103b48c-6111-4717-ad7a-b1daec6176e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:04.899232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103b48c-6111-4717-ad7a-b1daec6176e5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.899509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103b48c-6111-4717-ad7a-b1daec6176e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.899776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103b48c-6111-4717-ad7a-b1daec6176e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.900474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103b48c-6111-4717-ad7a-b1daec6176e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:04.900781 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103b48c-6111-4717-ad7a-b1daec6176e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:04.902299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103b48c-6111-4717-ad7a-b1daec6176e5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:04.921620 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 346/681] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:04.923351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103b48c-6111-4717-ad7a-b1daec6176e5 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:04.959012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:05.003918 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c1af5-40d5-4b66-bbad-663c8dc01ea3 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:05.048859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d16b61-ce17-4368-a17a-3869651d0788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:05.049751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d16b61-ce17-4368-a17a-3869651d0788 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:05.050123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d16b61-ce17-4368-a17a-3869651d0788 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:05.050499 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d16b61-ce17-4368-a17a-3869651d0788 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:05.051313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d16b61-ce17-4368-a17a-3869651d0788 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:05.052848 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d16b61-ce17-4368-a17a-3869651d0788 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-06d16b61-ce17-4368-a17a-3869651d0788 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:05.066707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:05.068699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:05.068699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:05.068699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:05.068699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:05.068699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:05.068699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:05.068699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:05.068699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:05.068699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:05.068699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:05.068699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:05.068699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:05.068699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:05.068699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:05.068699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:05.068699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:05.070770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:05.070770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:05.070770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:05.070770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:05.070770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:05.070770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:05.070770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:05.070770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:05.070770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:05.070770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:05.070770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:05.070770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:05.070770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:05.070770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:05.070770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 336/682] 10.222.0.88 () {56 vars in 1129 bytes} [Tue Aug 23 22:10:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:05.073099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d16b61-ce17-4368-a17a-3869651d0788 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:05.075846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103b48c-6111-4717-ad7a-b1daec6176e5 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:05.124145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6213d-eabb-4049-a781-ae6707f9b825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:05.124668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6213d-eabb-4049-a781-ae6707f9b825 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:05.124954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6213d-eabb-4049-a781-ae6707f9b825 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:05.124954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6213d-eabb-4049-a781-ae6707f9b825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:05.126950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d6213d-eabb-4049-a781-ae6707f9b825 None admin] RBAC: Adding query filter params (role.id=23ea214daa734346a6da3a0a8c50d23a, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=e3517b3b077a4a1fa76163657b46930c) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:05.127350 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d6213d-eabb-4049-a781-ae6707f9b825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:05.128943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d6213d-eabb-4049-a781-ae6707f9b825 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:05.136496 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 347/683] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:04 2022] GET /identity/v3/role_assignments?user.id=e3517b3b077a4a1fa76163657b46930c&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=23ea214daa734346a6da3a0a8c50d23a => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:05.137687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6213d-eabb-4049-a781-ae6707f9b825 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:05.205156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:05.235542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d16b61-ce17-4368-a17a-3869651d0788 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:05.282543 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a1498-ede3-4d09-bc94-6895f766a9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:05.283582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a1498-ede3-4d09-bc94-6895f766a9da None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:05.284025 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a1498-ede3-4d09-bc94-6895f766a9da None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:05.284430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a1498-ede3-4d09-bc94-6895f766a9da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:05.293812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a1498-ede3-4d09-bc94-6895f766a9da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:05.295619 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a1498-ede3-4d09-bc94-6895f766a9da None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:05.299262 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:05.313187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-cf5a1498-ede3-4d09-bc94-6895f766a9da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:05.313187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:05.313187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:05.313187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:05.313187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:05.313187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:05.313187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:05.313187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:05.313187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:05.313187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:05.313187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:05.313187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:05.313187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:05.313187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:05.313187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:05.315413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:05.318355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:05.318355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:05.318355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:05.318355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:05.318355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:05.318355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:05.318355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:05.318355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:05.318355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 337/684] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:05.318355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a1498-ede3-4d09-bc94-6895f766a9da None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:05.340030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6213d-eabb-4049-a781-ae6707f9b825 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:05.397839 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4230cf-2a35-4c95-b954-90d52e87190f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:05.398312 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4230cf-2a35-4c95-b954-90d52e87190f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:05.399135 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4230cf-2a35-4c95-b954-90d52e87190f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:05.399135 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4230cf-2a35-4c95-b954-90d52e87190f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:05.466493 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a1498-ede3-4d09-bc94-6895f766a9da None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:05.496901 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4230cf-2a35-4c95-b954-90d52e87190f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:05.498883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4230cf-2a35-4c95-b954-90d52e87190f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:05.528961 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9968832-5a1e-4fcc-acd1-07efcc2e2f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:05.530142 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9968832-5a1e-4fcc-acd1-07efcc2e2f72 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:05.531138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9968832-5a1e-4fcc-acd1-07efcc2e2f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:05.531138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9968832-5a1e-4fcc-acd1-07efcc2e2f72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:05.533911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9968832-5a1e-4fcc-acd1-07efcc2e2f72 None admin] RBAC: Adding query filter params (role.id=23ea214daa734346a6da3a0a8c50d23a, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=6505100494d4496591407c48b48535ee) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:05.534560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9968832-5a1e-4fcc-acd1-07efcc2e2f72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:05.537391 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9968832-5a1e-4fcc-acd1-07efcc2e2f72 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:05.556519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 338/685] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:05 2022] GET /identity/v3/role_assignments?user.id=6505100494d4496591407c48b48535ee&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=23ea214daa734346a6da3a0a8c50d23a => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:05.556519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9968832-5a1e-4fcc-acd1-07efcc2e2f72 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:05.561833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:05.626738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:05.653571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 348/686] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:05.654716 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4230cf-2a35-4c95-b954-90d52e87190f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:05.759413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9968832-5a1e-4fcc-acd1-07efcc2e2f72 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:05.815283 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4230cf-2a35-4c95-b954-90d52e87190f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:05.854894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796397d-7e91-4b1e-a2af-ec9ce66302ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:05.855714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796397d-7e91-4b1e-a2af-ec9ce66302ba None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:05.855858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796397d-7e91-4b1e-a2af-ec9ce66302ba None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:05.856037 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796397d-7e91-4b1e-a2af-ec9ce66302ba None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:05.856678 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796397d-7e91-4b1e-a2af-ec9ce66302ba None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:05.856984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796397d-7e91-4b1e-a2af-ec9ce66302ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:05.858410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796397d-7e91-4b1e-a2af-ec9ce66302ba None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:05.867612 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e52e1-dd19-4e2d-9759-aa131dae7be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:05.868335 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e52e1-dd19-4e2d-9759-aa131dae7be1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:05.868465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 339/687] 10.222.0.88 () {56 vars in 1124 bytes} [Tue Aug 23 22:10:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:05.868631 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e52e1-dd19-4e2d-9759-aa131dae7be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:05.868786 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e52e1-dd19-4e2d-9759-aa131dae7be1 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:05.869470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e52e1-dd19-4e2d-9759-aa131dae7be1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:05.870821 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e52e1-dd19-4e2d-9759-aa131dae7be1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:05.870988 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796397d-7e91-4b1e-a2af-ec9ce66302ba None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:05.871286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e52e1-dd19-4e2d-9759-aa131dae7be1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:05.892505 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 349/688] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:05.898223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e52e1-dd19-4e2d-9759-aa131dae7be1 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:05.962908 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:06.015450 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:06.058302 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e52e1-dd19-4e2d-9759-aa131dae7be1 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:06.085317 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796397d-7e91-4b1e-a2af-ec9ce66302ba None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:06.125881 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b766728-0fef-44cc-9f02-4257d28467f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:06.127294 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b766728-0fef-44cc-9f02-4257d28467f5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:06.127990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b766728-0fef-44cc-9f02-4257d28467f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:06.128691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b766728-0fef-44cc-9f02-4257d28467f5 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:06.130087 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b766728-0fef-44cc-9f02-4257d28467f5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:06.131035 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b766728-0fef-44cc-9f02-4257d28467f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:06.134059 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b766728-0fef-44cc-9f02-4257d28467f5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:06.151611 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 350/689] 10.222.0.88 () {56 vars in 1124 bytes} [Tue Aug 23 22:10:05 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:06.152751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed98dd-91c3-4cbe-935d-096020456778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:06.153984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed98dd-91c3-4cbe-935d-096020456778 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:06.154274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b766728-0fef-44cc-9f02-4257d28467f5 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:06.156811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed98dd-91c3-4cbe-935d-096020456778 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:06.157728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed98dd-91c3-4cbe-935d-096020456778 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:06.177798 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed98dd-91c3-4cbe-935d-096020456778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:06.180473 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed98dd-91c3-4cbe-935d-096020456778 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:06.201819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-9fed98dd-91c3-4cbe-935d-096020456778 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:06.201819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:06.201819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:06.201819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:06.201819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:06.201819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:06.201819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:06.201819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:06.201819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:06.201819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:06.201819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:06.201819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:06.201819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:06.201819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:06.201819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:06.201819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:06.203909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:06.206250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:06.206250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:06.206250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:06.206250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 340/690] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:06.214880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed98dd-91c3-4cbe-935d-096020456778 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:06.411163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed98dd-91c3-4cbe-935d-096020456778 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:06.457103 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:06.505864 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1f841-08a2-44c7-a537-152427942767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:06.507638 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1f841-08a2-44c7-a537-152427942767 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:06.508133 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1f841-08a2-44c7-a537-152427942767 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:06.508612 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1f841-08a2-44c7-a537-152427942767 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:06.509806 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1f841-08a2-44c7-a537-152427942767 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:06.510341 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1f841-08a2-44c7-a537-152427942767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:06.510895 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b766728-0fef-44cc-9f02-4257d28467f5 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:06.512885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1f841-08a2-44c7-a537-152427942767 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:06.536395 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:06.549584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 341/691] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:06.556018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1f841-08a2-44c7-a537-152427942767 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:06.588920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ab68e-69aa-41db-b09b-a01e68b53784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:06.592565 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ab68e-69aa-41db-b09b-a01e68b53784 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:06.592916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ab68e-69aa-41db-b09b-a01e68b53784 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:06.593356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ab68e-69aa-41db-b09b-a01e68b53784 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:06.595083 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ab68e-69aa-41db-b09b-a01e68b53784 None admin] RBAC: Adding query filter params (role.id=23ea214daa734346a6da3a0a8c50d23a, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=2211e237fe554a82baf129e440a0fb43) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:06.595408 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ab68e-69aa-41db-b09b-a01e68b53784 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:06.597097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ab68e-69aa-41db-b09b-a01e68b53784 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:06.607713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 351/692] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:06 2022] GET /identity/v3/role_assignments?user.id=2211e237fe554a82baf129e440a0fb43&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=23ea214daa734346a6da3a0a8c50d23a => generated 608 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:06.786278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1f841-08a2-44c7-a537-152427942767 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:06.849111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f7f30-925d-4e6b-ba79-f5334f3b5417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:06.849111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f7f30-925d-4e6b-ba79-f5334f3b5417 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:06.849111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f7f30-925d-4e6b-ba79-f5334f3b5417 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:06.849111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f7f30-925d-4e6b-ba79-f5334f3b5417 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:06.864700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f7f30-925d-4e6b-ba79-f5334f3b5417 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:06.867517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f7f30-925d-4e6b-ba79-f5334f3b5417 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:06.891859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-185f7f30-925d-4e6b-ba79-f5334f3b5417 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:06.891859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:06.891859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:06.891859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:06.891859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:06.891859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:06.891859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:06.891859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:06.891859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:06.891859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:06.891859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:06.891859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:06.891859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:06.891859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:06.891859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:06.893618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:06.897113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:06.897113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:06.897113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:06.897113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:06.897113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:06.897113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:06.897113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:06.897113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:06.897113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 342/693] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:06.904560 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ab68e-69aa-41db-b09b-a01e68b53784 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:07.106455 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ab68e-69aa-41db-b09b-a01e68b53784 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:07.178764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7115b22-e372-4892-97a6-6f6964fa78ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:07.179793 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7115b22-e372-4892-97a6-6f6964fa78ec None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:07.180282 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7115b22-e372-4892-97a6-6f6964fa78ec None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:07.180764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7115b22-e372-4892-97a6-6f6964fa78ec None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:07.182048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7115b22-e372-4892-97a6-6f6964fa78ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:07.182591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7115b22-e372-4892-97a6-6f6964fa78ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:07.185335 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7115b22-e372-4892-97a6-6f6964fa78ec None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:07.212454 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 352/694] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:07.223090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f7f30-925d-4e6b-ba79-f5334f3b5417 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:07.325484 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:07.447829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f7f30-925d-4e6b-ba79-f5334f3b5417 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:07.522627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289edafd-e320-4f93-abfe-eb86bf182fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:07.523265 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289edafd-e320-4f93-abfe-eb86bf182fbc None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:07.523547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289edafd-e320-4f93-abfe-eb86bf182fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:07.523808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289edafd-e320-4f93-abfe-eb86bf182fbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:07.524635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289edafd-e320-4f93-abfe-eb86bf182fbc None admin] RBAC: Adding query filter params (role.id=23ea214daa734346a6da3a0a8c50d23a, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=601a3defca70428689ee22517d7bcb19) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:07.524951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289edafd-e320-4f93-abfe-eb86bf182fbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:07.526701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289edafd-e320-4f93-abfe-eb86bf182fbc None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:07.538126 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 343/695] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:07 2022] GET /identity/v3/role_assignments?user.id=601a3defca70428689ee22517d7bcb19&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=23ea214daa734346a6da3a0a8c50d23a => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:08.069166 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382646de-198c-40b5-9be2-71895828739e None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:08.069688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382646de-198c-40b5-9be2-71895828739e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:08.069688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382646de-198c-40b5-9be2-71895828739e None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:08.071419 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 353/696] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:08.078577 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44ef69-7e72-4f7d-8e96-af64bd4ce085 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:08.079286 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44ef69-7e72-4f7d-8e96-af64bd4ce085 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:08.079286 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44ef69-7e72-4f7d-8e96-af64bd4ce085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:08.101378 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-df44ef69-7e72-4f7d-8e96-af64bd4ce085 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:08.102651 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-df44ef69-7e72-4f7d-8e96-af64bd4ce085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:08.206092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 344/697] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:08.542195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382646de-198c-40b5-9be2-71895828739e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:08.600553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:08.711404 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382646de-198c-40b5-9be2-71895828739e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:08.777604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e830818-19a7-4008-b077-ccf45cd74d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:08.778282 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e830818-19a7-4008-b077-ccf45cd74d3c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:08.778595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e830818-19a7-4008-b077-ccf45cd74d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:08.778870 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e830818-19a7-4008-b077-ccf45cd74d3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:08.804342 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e830818-19a7-4008-b077-ccf45cd74d3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:08.810613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-9e830818-19a7-4008-b077-ccf45cd74d3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:08.810613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:08.810613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:08.810613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:08.810613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:08.810613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:08.810613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:08.810613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:08.810613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:08.810613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:08.810613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:08.810613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:08.810613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:08.810613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:08.810613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:08.810613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:08.811604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:08.811604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:08.811604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:08.811604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:08.811604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:08.811604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:08.811604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:08.811604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:08.811604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:08.811604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:08.811604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:08.811604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:08.811604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:08.811604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:08.811604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:08.811604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:08.812550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:08.812550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:08.812550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:08.812550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:08.812550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:08.812550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:08.812550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:08.812550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:08.812550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:08.812550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:08.812550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:08.812550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:08.817221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 354/698] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:08.821668 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44ef69-7e72-4f7d-8e96-af64bd4ce085 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:08.894885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f203990-aaf5-43b0-a873-4255c00ccb41 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:08.895359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f203990-aaf5-43b0-a873-4255c00ccb41 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:08.895782 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f203990-aaf5-43b0-a873-4255c00ccb41 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:08.900795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 355/699] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:08.908071 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935161aa-6071-4f4b-9cc5-d51488280352 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:08.908380 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935161aa-6071-4f4b-9cc5-d51488280352 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:08.908777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935161aa-6071-4f4b-9cc5-d51488280352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:08.917180 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:08.933733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-935161aa-6071-4f4b-9cc5-d51488280352 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:08.935755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-935161aa-6071-4f4b-9cc5-d51488280352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:09.061590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 356/700] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:10:09.112637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44ef69-7e72-4f7d-8e96-af64bd4ce085 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:09.179697 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3909010-2703-4a52-a5ca-0cec580477b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:09.180414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3909010-2703-4a52-a5ca-0cec580477b6 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:09.180694 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3909010-2703-4a52-a5ca-0cec580477b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:09.180974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3909010-2703-4a52-a5ca-0cec580477b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:09.181779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3909010-2703-4a52-a5ca-0cec580477b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:09.182222 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3909010-2703-4a52-a5ca-0cec580477b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:09.183882 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3909010-2703-4a52-a5ca-0cec580477b6 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:09.201315 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 345/701] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:09.208095 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935161aa-6071-4f4b-9cc5-d51488280352 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:09.247231 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:09.352244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935161aa-6071-4f4b-9cc5-d51488280352 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:09.398643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b38936-1e21-4d7b-93dc-a1fd0b83adb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:09.399727 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b38936-1e21-4d7b-93dc-a1fd0b83adb7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:09.399997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b38936-1e21-4d7b-93dc-a1fd0b83adb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:09.400284 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b38936-1e21-4d7b-93dc-a1fd0b83adb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:09.414926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b38936-1e21-4d7b-93dc-a1fd0b83adb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:09.417384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b38936-1e21-4d7b-93dc-a1fd0b83adb7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:09.436785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-64b38936-1e21-4d7b-93dc-a1fd0b83adb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:09.436785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:09.436785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:09.436785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:09.436785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:09.436785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:09.436785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:09.436785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:09.436785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:09.436785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:09.436785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:09.436785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:09.436785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:09.436785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:09.436785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:09.436785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:09.439037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:09.440834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:09.440834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:09.440834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:09.440834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 357/702] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:09.446273 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3909010-2703-4a52-a5ca-0cec580477b6 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:09.648998 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b38936-1e21-4d7b-93dc-a1fd0b83adb7 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:09.658277 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:09.666661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3909010-2703-4a52-a5ca-0cec580477b6 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:09.798651 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f2858-7b22-4073-b34b-0b336c6b32fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:09.799296 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f2858-7b22-4073-b34b-0b336c6b32fe None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:09.799605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f2858-7b22-4073-b34b-0b336c6b32fe None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:09.805108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f2858-7b22-4073-b34b-0b336c6b32fe None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:09.805108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f2858-7b22-4073-b34b-0b336c6b32fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:09.805108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f2858-7b22-4073-b34b-0b336c6b32fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:09.805108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f2858-7b22-4073-b34b-0b336c6b32fe None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:09.819005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:09.845255 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 346/703] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:09.850903 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f2858-7b22-4073-b34b-0b336c6b32fe None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:09.887910 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b38936-1e21-4d7b-93dc-a1fd0b83adb7 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:09.944289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef51fca-47a4-4023-b7df-3edd5723e3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:09.946157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef51fca-47a4-4023-b7df-3edd5723e3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:09.946630 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef51fca-47a4-4023-b7df-3edd5723e3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:09.947068 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef51fca-47a4-4023-b7df-3edd5723e3c0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:09.948102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef51fca-47a4-4023-b7df-3edd5723e3c0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:09.950804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef51fca-47a4-4023-b7df-3edd5723e3c0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-cef51fca-47a4-4023-b7df-3edd5723e3c0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:09.969295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:09.970413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:09.970413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:09.970413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:09.970413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:09.970413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:09.970413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:09.970413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:09.970413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:09.970413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:09.970413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:09.970413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:09.970413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:09.970413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:09.970413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:09.970413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:09.970413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:09.972437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:09.972437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:09.972437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:09.972437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:09.972437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:09.972437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:09.972437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:09.972437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:09.972437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:09.972437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:09.972437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:09.972437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:09.972437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:09.972437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:09.972437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 358/704] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:09 2022] GET /identity/v3/services/network => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:09.974317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c6c6e-5055-4544-bf95-cb522f402263 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:09.974317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c6c6e-5055-4544-bf95-cb522f402263 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:09.974317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c6c6e-5055-4544-bf95-cb522f402263 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:09.976391 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 359/705] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:09.980822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c6c6e-5055-4544-bf95-cb522f402263 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:10.061112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f2858-7b22-4073-b34b-0b336c6b32fe None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:10.113143 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504d91b-9c64-43fe-b842-ccc8eed4f145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:10.114299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504d91b-9c64-43fe-b842-ccc8eed4f145 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.114585 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504d91b-9c64-43fe-b842-ccc8eed4f145 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.114923 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504d91b-9c64-43fe-b842-ccc8eed4f145 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.125602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9504d91b-9c64-43fe-b842-ccc8eed4f145 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:10.126425 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c6c6e-5055-4544-bf95-cb522f402263 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:10.127279 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9504d91b-9c64-43fe-b842-ccc8eed4f145 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:10.143521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-9504d91b-9c64-43fe-b842-ccc8eed4f145 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:10.143521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:10.143521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:10.143521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:10.143521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:10.143521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:10.143521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:10.143521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:10.143521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:10.143521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:10.143521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:10.143521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:10.143521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:10.143521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:10.143521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:10.143521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:10.144565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:10.145560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:10.145560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:10.145560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:10.145560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:10.145560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:10.145560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:10.145560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:10.145560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:10.145560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:10.145560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:10.149132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 347/706] 10.222.0.88 () {56 vars in 1093 bytes} [Tue Aug 23 22:10:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:10.150955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eb639-f0fa-4023-bf40-7684bcb2a627 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.151245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eb639-f0fa-4023-bf40-7684bcb2a627 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.151521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eb639-f0fa-4023-bf40-7684bcb2a627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.174896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-d15eb639-f0fa-4023-bf40-7684bcb2a627 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:10.176224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d15eb639-f0fa-4023-bf40-7684bcb2a627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:10.177691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13951518-05be-4160-ad18-5a0904513ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:10.178857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13951518-05be-4160-ad18-5a0904513ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.179387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13951518-05be-4160-ad18-5a0904513ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.180880 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13951518-05be-4160-ad18-5a0904513ab1 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.181772 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13951518-05be-4160-ad18-5a0904513ab1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:10.182286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13951518-05be-4160-ad18-5a0904513ab1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:10.183909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13951518-05be-4160-ad18-5a0904513ab1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:10.190824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:10.192479 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 360/707] 10.222.0.88 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:10.193626 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13951518-05be-4160-ad18-5a0904513ab1 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:10.211236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:10.291321 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 348/708] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:10.292410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a8f59-9db3-4d18-a46a-e737400d2f6c None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.292827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a8f59-9db3-4d18-a46a-e737400d2f6c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.293181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a8f59-9db3-4d18-a46a-e737400d2f6c None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.295176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 349/709] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:10.296067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a8f59-9db3-4d18-a46a-e737400d2f6c None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:10.383732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13951518-05be-4160-ad18-5a0904513ab1 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:10.449011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497b624-651d-4839-bdfd-d0455064fc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:10.449661 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497b624-651d-4839-bdfd-d0455064fc2c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.449945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497b624-651d-4839-bdfd-d0455064fc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.450229 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497b624-651d-4839-bdfd-d0455064fc2c None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.451133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497b624-651d-4839-bdfd-d0455064fc2c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:10.451895 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497b624-651d-4839-bdfd-d0455064fc2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:10.454384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497b624-651d-4839-bdfd-d0455064fc2c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:10.478239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a8f59-9db3-4d18-a46a-e737400d2f6c None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:10.478636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 361/710] 10.222.0.88 () {56 vars in 1133 bytes} [Tue Aug 23 22:10:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:10.479990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af286c-c8a7-4e50-a13c-443246969881 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.480284 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af286c-c8a7-4e50-a13c-443246969881 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.480561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af286c-c8a7-4e50-a13c-443246969881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.504466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b0af286c-c8a7-4e50-a13c-443246969881 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:10.506056 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b0af286c-c8a7-4e50-a13c-443246969881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:10.528360 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d7b64-5596-4e27-84d7-8babcd58ca09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:10.529150 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d7b64-5596-4e27-84d7-8babcd58ca09 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.529449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d7b64-5596-4e27-84d7-8babcd58ca09 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.529833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d7b64-5596-4e27-84d7-8babcd58ca09 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.530547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d7b64-5596-4e27-84d7-8babcd58ca09 None admin] RBAC: Adding query filter params (type=network) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:10.531580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d7b64-5596-4e27-84d7-8babcd58ca09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:10.536337 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d7b64-5596-4e27-84d7-8babcd58ca09 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:10.539845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 350/711] 10.222.0.88 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:10.540809 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d7b64-5596-4e27-84d7-8babcd58ca09 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:10.572622 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:10.642292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:10.666270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 362/712] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:10.667491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af286c-c8a7-4e50-a13c-443246969881 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:10.776182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d7b64-5596-4e27-84d7-8babcd58ca09 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:10.838519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af286c-c8a7-4e50-a13c-443246969881 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:10.852397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f08ed-268e-4b63-93b0-43a530dda450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:10.853161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f08ed-268e-4b63-93b0-43a530dda450 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.853592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f08ed-268e-4b63-93b0-43a530dda450 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.854048 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f08ed-268e-4b63-93b0-43a530dda450 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.925513 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baef73e-a00b-49e9-97f6-a304c3b47035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:10.926164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baef73e-a00b-49e9-97f6-a304c3b47035 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.926445 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baef73e-a00b-49e9-97f6-a304c3b47035 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.926725 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baef73e-a00b-49e9-97f6-a304c3b47035 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.927367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baef73e-a00b-49e9-97f6-a304c3b47035 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:10.929625 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baef73e-a00b-49e9-97f6-a304c3b47035 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:10.970861 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 363/713] 10.222.0.88 () {60 vars in 1141 bytes} [Tue Aug 23 22:10:10 2022] POST /identity/v3/services => generated 251 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:11.019771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f08ed-268e-4b63-93b0-43a530dda450 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:11.021477 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f08ed-268e-4b63-93b0-43a530dda450 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:11.035799 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e9a8a-7fd4-4de2-8a97-86929bca6afa None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:11.036714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e9a8a-7fd4-4de2-8a97-86929bca6afa None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:11.040200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e9a8a-7fd4-4de2-8a97-86929bca6afa None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:11.044928 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 364/714] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:11 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:11.050786 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175571ea-dcf0-40d0-8e82-6b39b8f3f8e1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:11.052800 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175571ea-dcf0-40d0-8e82-6b39b8f3f8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:11.053387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175571ea-dcf0-40d0-8e82-6b39b8f3f8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:11.056902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:11.107528 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-175571ea-dcf0-40d0-8e82-6b39b8f3f8e1 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:11.112075 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-175571ea-dcf0-40d0-8e82-6b39b8f3f8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:11.208382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:11.216035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 351/715] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:11.217981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f08ed-268e-4b63-93b0-43a530dda450 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:11.228356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 365/716] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:11 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:10:11.229418 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175571ea-dcf0-40d0-8e82-6b39b8f3f8e1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:11.343685 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f08ed-268e-4b63-93b0-43a530dda450 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:11.356491 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:11.407695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b7ceb-ec7f-40be-8cc7-b1a41261ba5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:11.409150 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b7ceb-ec7f-40be-8cc7-b1a41261ba5e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:11.409979 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b7ceb-ec7f-40be-8cc7-b1a41261ba5e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:11.410744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b7ceb-ec7f-40be-8cc7-b1a41261ba5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:11.413755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175571ea-dcf0-40d0-8e82-6b39b8f3f8e1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:11.431408 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b7ceb-ec7f-40be-8cc7-b1a41261ba5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:11.434845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b7ceb-ec7f-40be-8cc7-b1a41261ba5e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:11.452946 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f849b-0d09-4466-a2a1-27ca9b863943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:11.454276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f849b-0d09-4466-a2a1-27ca9b863943 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:11.454806 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f849b-0d09-4466-a2a1-27ca9b863943 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:11.455147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f849b-0d09-4466-a2a1-27ca9b863943 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:11.466499 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f849b-0d09-4466-a2a1-27ca9b863943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:11.467397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-d35b7ceb-ec7f-40be-8cc7-b1a41261ba5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:11.467397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:11.467397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:11.467397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:11.467397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:11.467397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:11.467397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:11.467397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:11.467397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:11.467397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:11.467397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:11.467397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:11.467397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:11.467397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:11.467397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:11.467397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:11.469355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:11.471137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:11.471137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:11.471137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:11.471458 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f849b-0d09-4466-a2a1-27ca9b863943 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:11.474999 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 352/717] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:11.480911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b7ceb-ec7f-40be-8cc7-b1a41261ba5e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:11.485598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-819f849b-0d09-4466-a2a1-27ca9b863943 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:11.485598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:11.485598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:11.485598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:11.485598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:11.485598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:11.485598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:11.485598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:11.485598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:11.485598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:11.485598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:11.485598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:11.485598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:11.485598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:11.485598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:11.485598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:11.487892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:11.489847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:11.489847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:11.489847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:11.489847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 366/718] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:11.492150 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f849b-0d09-4466-a2a1-27ca9b863943 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:11.528046 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:11.616959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f849b-0d09-4466-a2a1-27ca9b863943 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:11.660603 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6db91-63e1-4121-a088-82b965cbc1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:11.661891 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6db91-63e1-4121-a088-82b965cbc1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:11.661891 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6db91-63e1-4121-a088-82b965cbc1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:11.662361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6db91-63e1-4121-a088-82b965cbc1f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:11.663265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6db91-63e1-4121-a088-82b965cbc1f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:11.663681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6db91-63e1-4121-a088-82b965cbc1f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:11.665313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6db91-63e1-4121-a088-82b965cbc1f6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:11.681709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 367/719] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:11.682840 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6db91-63e1-4121-a088-82b965cbc1f6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:11.693882 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b7ceb-ec7f-40be-8cc7-b1a41261ba5e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:11.760677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920ff24-00ec-4567-8767-bd77debb8f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:11.771115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920ff24-00ec-4567-8767-bd77debb8f5b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:11.772041 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920ff24-00ec-4567-8767-bd77debb8f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:11.772854 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920ff24-00ec-4567-8767-bd77debb8f5b None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:11.774340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920ff24-00ec-4567-8767-bd77debb8f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:11.775128 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920ff24-00ec-4567-8767-bd77debb8f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:11.777932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920ff24-00ec-4567-8767-bd77debb8f5b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:11.800615 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:11.813264 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6db91-63e1-4121-a088-82b965cbc1f6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:11.814114 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 353/720] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:11.816147 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920ff24-00ec-4567-8767-bd77debb8f5b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:11.854374 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90ae12-1375-4648-9232-f611a3cd3d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:11.855045 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90ae12-1375-4648-9232-f611a3cd3d8d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:11.855325 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90ae12-1375-4648-9232-f611a3cd3d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:11.855588 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90ae12-1375-4648-9232-f611a3cd3d8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:11.864515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:11.870031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90ae12-1375-4648-9232-f611a3cd3d8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:11.876079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-fd90ae12-1375-4648-9232-f611a3cd3d8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:11.876079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:11.876079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:11.876079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:11.876079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:11.876079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:11.876079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:11.876079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:11.876079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:11.876079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:11.876079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:11.876079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:11.876079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:11.876079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:11.876079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:11.876079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:11.878627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:11.878627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:11.878627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:11.878627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:11.878627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:11.878627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:11.878627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:11.878627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:11.878627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:11.878627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:11.878627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:11.878627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:11.878627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:11.878627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:11.878627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:11.878627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:11.880238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:11.880238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:11.880238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:11.880238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:11.880238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:11.880238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:11.880238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:11.880238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:11.880238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:11.880238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:11.880238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:11.880238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:11.880238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 368/721] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:11.880238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90ae12-1375-4648-9232-f611a3cd3d8d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:11.995467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920ff24-00ec-4567-8767-bd77debb8f5b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:12.020577 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90ae12-1375-4648-9232-f611a3cd3d8d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:12.039257 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae09c0-e3d4-4484-8f4a-b6f44e1bc956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:12.040381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae09c0-e3d4-4484-8f4a-b6f44e1bc956 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:12.040966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae09c0-e3d4-4484-8f4a-b6f44e1bc956 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:12.041493 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae09c0-e3d4-4484-8f4a-b6f44e1bc956 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:12.050558 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae09c0-e3d4-4484-8f4a-b6f44e1bc956 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:12.052582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae09c0-e3d4-4484-8f4a-b6f44e1bc956 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:12.066951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-f5ae09c0-e3d4-4484-8f4a-b6f44e1bc956 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:12.066951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:12.066951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:12.066951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:12.066951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:12.066951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:12.066951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:12.066951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:12.066951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:12.066951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:12.066951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:12.066951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:12.066951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:12.066951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:12.066951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:12.067925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:12.068825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:12.068825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:12.068825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:12.068825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:12.068825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:12.068825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:12.068825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:12.068825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:12.071524 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 354/722] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:12.072749 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae09c0-e3d4-4484-8f4a-b6f44e1bc956 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:12.084693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d7aef-607a-44c5-a550-d2c25ca9fdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:12.085724 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d7aef-607a-44c5-a550-d2c25ca9fdcd None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:12.086436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d7aef-607a-44c5-a550-d2c25ca9fdcd None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:12.086796 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d7aef-607a-44c5-a550-d2c25ca9fdcd None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:12.088303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d7aef-607a-44c5-a550-d2c25ca9fdcd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:12.090789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d7aef-607a-44c5-a550-d2c25ca9fdcd None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:12.131561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 369/723] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:10:11 2022] POST /identity/v3/users => generated 254 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:12.133225 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d7aef-607a-44c5-a550-d2c25ca9fdcd None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:12.153457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:12.178589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:12.205083 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae09c0-e3d4-4484-8f4a-b6f44e1bc956 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:12.249107 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102ae80-4f4f-42b6-b206-058ca20ea36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:12.250764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102ae80-4f4f-42b6-b206-058ca20ea36b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:12.251352 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102ae80-4f4f-42b6-b206-058ca20ea36b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:12.251887 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102ae80-4f4f-42b6-b206-058ca20ea36b None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:12.252866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102ae80-4f4f-42b6-b206-058ca20ea36b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:12.253403 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102ae80-4f4f-42b6-b206-058ca20ea36b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:12.255223 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102ae80-4f4f-42b6-b206-058ca20ea36b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:12.266269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 355/724] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:12.273541 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102ae80-4f4f-42b6-b206-058ca20ea36b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:12.353510 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d7aef-607a-44c5-a550-d2c25ca9fdcd None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:12.422168 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47f28a-938b-4626-bada-a7eedbf324d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:12.423752 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47f28a-938b-4626-bada-a7eedbf324d8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:12.424468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47f28a-938b-4626-bada-a7eedbf324d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:12.424953 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47f28a-938b-4626-bada-a7eedbf324d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:12.429298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47f28a-938b-4626-bada-a7eedbf324d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:12.430266 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47f28a-938b-4626-bada-a7eedbf324d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:12.434211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47f28a-938b-4626-bada-a7eedbf324d8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:12.466619 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 370/725] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:12.466875 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102ae80-4f4f-42b6-b206-058ca20ea36b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:12.476008 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47f28a-938b-4626-bada-a7eedbf324d8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:12.532319 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e5d72-36b5-4424-b1c5-99362771301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:12.533524 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e5d72-36b5-4424-b1c5-99362771301b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:12.534033 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e5d72-36b5-4424-b1c5-99362771301b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:12.534536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e5d72-36b5-4424-b1c5-99362771301b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:12.562949 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e5d72-36b5-4424-b1c5-99362771301b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:12.566958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e5d72-36b5-4424-b1c5-99362771301b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:12.574188 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:12.585405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:12.610016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-c47e5d72-36b5-4424-b1c5-99362771301b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:12.610016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:12.610016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:12.610016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:12.610016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:12.610016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:12.610016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:12.610016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:12.610016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:12.610016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:12.610016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:12.610016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:12.610016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:12.610016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:12.610016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:12.610016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:12.611020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:12.612556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:12.612556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:12.612556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:12.612556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 356/726] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:12.622147 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e5d72-36b5-4424-b1c5-99362771301b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:12.651673 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47f28a-938b-4626-bada-a7eedbf324d8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:12.721521 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79770e5-ad0e-4c4e-9790-e9cada99965b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:12.722560 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79770e5-ad0e-4c4e-9790-e9cada99965b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:12.723040 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79770e5-ad0e-4c4e-9790-e9cada99965b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:12.723541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79770e5-ad0e-4c4e-9790-e9cada99965b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:12.724982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79770e5-ad0e-4c4e-9790-e9cada99965b None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=e3517b3b077a4a1fa76163657b46930c) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:12.726896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79770e5-ad0e-4c4e-9790-e9cada99965b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:12.729397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79770e5-ad0e-4c4e-9790-e9cada99965b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:12.741771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 371/727] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:12 2022] GET /identity/v3/role_assignments?user.id=e3517b3b077a4a1fa76163657b46930c&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=dc5d71ab685e420697f520aa3a368177 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:12.831138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e5d72-36b5-4424-b1c5-99362771301b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:12.895485 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085f235-ac39-4a7e-940d-ae4fc8f45a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:12.896583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085f235-ac39-4a7e-940d-ae4fc8f45a01 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:12.897014 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085f235-ac39-4a7e-940d-ae4fc8f45a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:12.897464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085f235-ac39-4a7e-940d-ae4fc8f45a01 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:12.898385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085f235-ac39-4a7e-940d-ae4fc8f45a01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:12.898989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085f235-ac39-4a7e-940d-ae4fc8f45a01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:12.900600 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085f235-ac39-4a7e-940d-ae4fc8f45a01 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:12.916703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 357/728] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:12.924047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79770e5-ad0e-4c4e-9790-e9cada99965b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:13.058830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdf9cd-cfd9-482b-ad5b-301d02713db7 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:13.059393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdf9cd-cfd9-482b-ad5b-301d02713db7 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:13.059986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdf9cd-cfd9-482b-ad5b-301d02713db7 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:13.062968 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 358/729] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:13.070189 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8085a-0ce1-438d-89f7-935497ea234e None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:13.070697 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8085a-0ce1-438d-89f7-935497ea234e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:13.071204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8085a-0ce1-438d-89f7-935497ea234e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:13.099351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79770e5-ad0e-4c4e-9790-e9cada99965b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:13.116729 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-43c8085a-0ce1-438d-89f7-935497ea234e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:13.118030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-43c8085a-0ce1-438d-89f7-935497ea234e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:13.170656 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:13.198259 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e6310-3dfe-42cd-bd54-9af8e799f976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:13.199331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e6310-3dfe-42cd-bd54-9af8e799f976 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:13.199817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e6310-3dfe-42cd-bd54-9af8e799f976 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:13.200281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e6310-3dfe-42cd-bd54-9af8e799f976 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:13.207887 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:13.228232 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 359/730] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:13.242981 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e6310-3dfe-42cd-bd54-9af8e799f976 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:13.245580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e6310-3dfe-42cd-bd54-9af8e799f976 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:13.284987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-c16e6310-3dfe-42cd-bd54-9af8e799f976 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:13.284987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:13.284987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:13.284987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:13.284987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:13.284987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:13.284987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:13.284987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:13.284987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:13.284987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:13.284987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:13.284987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:13.284987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:13.284987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:13.284987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:13.284987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:13.286889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:13.288779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:13.288779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:13.288779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:13.288779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:13.288779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:13.288779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:13.288779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:13.288779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:13.288779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:13.288779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:13.293665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 372/731] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:13.302227 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8085a-0ce1-438d-89f7-935497ea234e None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:13.552787 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8085a-0ce1-438d-89f7-935497ea234e None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:13.615740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad9668-8fb3-431d-b518-12e6611a960e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:13.616978 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad9668-8fb3-431d-b518-12e6611a960e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:13.617549 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad9668-8fb3-431d-b518-12e6611a960e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:13.618141 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad9668-8fb3-431d-b518-12e6611a960e None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:13.619141 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad9668-8fb3-431d-b518-12e6611a960e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:13.619654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad9668-8fb3-431d-b518-12e6611a960e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:13.621553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad9668-8fb3-431d-b518-12e6611a960e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:13.651737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 360/732] 10.222.0.88 () {56 vars in 1135 bytes} [Tue Aug 23 22:10:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:13.661265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e6310-3dfe-42cd-bd54-9af8e799f976 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:13.715064 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad9668-8fb3-431d-b518-12e6611a960e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:13.793523 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:13.861234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e6310-3dfe-42cd-bd54-9af8e799f976 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:13.886533 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad9668-8fb3-431d-b518-12e6611a960e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:13.902663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa28d4-3154-4068-a265-adc75714363a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:13.903708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa28d4-3154-4068-a265-adc75714363a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:13.904193 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa28d4-3154-4068-a265-adc75714363a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:13.904726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa28d4-3154-4068-a265-adc75714363a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:13.944489 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3290753-8137-4262-b222-fe9d34cb2b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:13.945134 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3290753-8137-4262-b222-fe9d34cb2b5c None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:13.945419 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3290753-8137-4262-b222-fe9d34cb2b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:13.945681 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3290753-8137-4262-b222-fe9d34cb2b5c None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:13.946371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3290753-8137-4262-b222-fe9d34cb2b5c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:13.947754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3290753-8137-4262-b222-fe9d34cb2b5c None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:13.981170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 361/733] 10.222.0.88 () {60 vars in 1141 bytes} [Tue Aug 23 22:10:13 2022] POST /identity/v3/services => generated 240 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:13.981170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6c404-399d-4146-bf05-e0aa5a9919d6 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:13.981170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6c404-399d-4146-bf05-e0aa5a9919d6 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:13.981170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6c404-399d-4146-bf05-e0aa5a9919d6 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:13.983717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 362/734] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:13.986834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fef83d-fcef-451a-9f25-a6cf22cbd6ff None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:13.987145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fef83d-fcef-451a-9f25-a6cf22cbd6ff None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:13.987384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fef83d-fcef-451a-9f25-a6cf22cbd6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:14.009584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-d3fef83d-fcef-451a-9f25-a6cf22cbd6ff None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:14.010871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d3fef83d-fcef-451a-9f25-a6cf22cbd6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:14.037346 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:14.070281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa28d4-3154-4068-a265-adc75714363a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:14.073103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa28d4-3154-4068-a265-adc75714363a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:14.126313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 363/735] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:14.322877 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 373/736] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:13 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:14.333049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fef83d-fcef-451a-9f25-a6cf22cbd6ff None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:14.395779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:14.463491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa28d4-3154-4068-a265-adc75714363a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:14.555734 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fef83d-fcef-451a-9f25-a6cf22cbd6ff None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:14.641567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c709059-7fa1-4bcd-9cc1-8638802a4ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:14.644828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c709059-7fa1-4bcd-9cc1-8638802a4ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:14.646052 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c709059-7fa1-4bcd-9cc1-8638802a4ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:14.649461 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c709059-7fa1-4bcd-9cc1-8638802a4ab7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:14.687827 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:14.693244 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c709059-7fa1-4bcd-9cc1-8638802a4ab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:14.696206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c709059-7fa1-4bcd-9cc1-8638802a4ab7 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:14.712479 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa28d4-3154-4068-a265-adc75714363a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:14.735451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-1c709059-7fa1-4bcd-9cc1-8638802a4ab7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:14.735451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:14.735451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:14.735451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:14.735451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:14.735451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:14.735451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:14.735451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:14.735451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:14.735451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:14.735451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:14.735451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:14.735451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:14.735451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:14.735451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:14.735451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:14.736614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:14.738129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:14.738129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:14.738129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:14.740521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 364/737] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:14.747404 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c709059-7fa1-4bcd-9cc1-8638802a4ab7 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:14.809091 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324493a-154a-472b-8d7b-e41db0507ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:14.809091 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324493a-154a-472b-8d7b-e41db0507ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:14.809091 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324493a-154a-472b-8d7b-e41db0507ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:14.809091 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324493a-154a-472b-8d7b-e41db0507ec2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:14.809559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324493a-154a-472b-8d7b-e41db0507ec2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:14.816487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324493a-154a-472b-8d7b-e41db0507ec2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-e324493a-154a-472b-8d7b-e41db0507ec2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:14.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:14.852974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:14.852974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:14.852974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:14.852974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:14.852974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:14.852974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:14.852974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:14.852974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:14.852974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:14.852974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:14.852974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:14.852974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:14.852974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:14.852974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:14.852974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:14.852974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:14.853903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:14.853903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:14.853903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:14.853903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:14.853903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:14.853903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:14.853903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:14.853903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:14.853903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:14.853903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:14.853903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:14.853903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:14.853903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:14.853903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:14.862508 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 374/738] 10.222.0.88 () {56 vars in 1129 bytes} [Tue Aug 23 22:10:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:14.870805 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324493a-154a-472b-8d7b-e41db0507ec2 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:15.011852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:15.066790 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c709059-7fa1-4bcd-9cc1-8638802a4ab7 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:15.165517 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324493a-154a-472b-8d7b-e41db0507ec2 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:15.175581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29373e97-39c7-4c47-9535-388cf8a8c663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:15.176663 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29373e97-39c7-4c47-9535-388cf8a8c663 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.177176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29373e97-39c7-4c47-9535-388cf8a8c663 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.177672 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29373e97-39c7-4c47-9535-388cf8a8c663 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.178894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29373e97-39c7-4c47-9535-388cf8a8c663 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:15.179454 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29373e97-39c7-4c47-9535-388cf8a8c663 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:15.185098 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29373e97-39c7-4c47-9535-388cf8a8c663 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:15.215962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 365/739] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:15.223425 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29373e97-39c7-4c47-9535-388cf8a8c663 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:15.279379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf78207-daad-4bd8-822b-64786ff0e375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:15.280377 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf78207-daad-4bd8-822b-64786ff0e375 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.280950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf78207-daad-4bd8-822b-64786ff0e375 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.281494 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf78207-daad-4bd8-822b-64786ff0e375 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.282682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf78207-daad-4bd8-822b-64786ff0e375 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:15.283256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf78207-daad-4bd8-822b-64786ff0e375 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:15.285919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf78207-daad-4bd8-822b-64786ff0e375 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:15.304747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 375/740] 10.222.0.88 () {56 vars in 1124 bytes} [Tue Aug 23 22:10:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:15.314555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf78207-daad-4bd8-822b-64786ff0e375 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:15.370477 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:15.495301 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29373e97-39c7-4c47-9535-388cf8a8c663 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:15.603777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf78207-daad-4bd8-822b-64786ff0e375 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:15.617869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec33fd-27f1-46d7-8702-d57b98e63736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:15.619534 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec33fd-27f1-46d7-8702-d57b98e63736 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.620282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec33fd-27f1-46d7-8702-d57b98e63736 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.621104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec33fd-27f1-46d7-8702-d57b98e63736 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.646056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec33fd-27f1-46d7-8702-d57b98e63736 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:15.649424 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec33fd-27f1-46d7-8702-d57b98e63736 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:15.669340 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bcd4a1-7657-4fc3-8b8b-0607095b0991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:15.670279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcd4a1-7657-4fc3-8b8b-0607095b0991 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.670713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcd4a1-7657-4fc3-8b8b-0607095b0991 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.671135 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcd4a1-7657-4fc3-8b8b-0607095b0991 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.672200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bcd4a1-7657-4fc3-8b8b-0607095b0991 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:15.672673 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bcd4a1-7657-4fc3-8b8b-0607095b0991 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:15.675018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bcd4a1-7657-4fc3-8b8b-0607095b0991 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:15.676075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-83ec33fd-27f1-46d7-8702-d57b98e63736 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:15.676075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:15.676075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:15.676075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:15.676075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:15.676075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:15.676075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:15.676075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:15.676075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:15.676075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:15.676075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:15.676075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:15.676075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:15.676075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:15.676075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:15.677742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:15.679530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:15.679530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:15.679530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:15.679530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:15.679530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:15.679530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:15.679530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:15.679530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:15.683395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 366/741] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 461 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:15.686103 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5981d13-90f0-49cb-a4dd-291440c03058 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.686818 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5981d13-90f0-49cb-a4dd-291440c03058 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.687783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5981d13-90f0-49cb-a4dd-291440c03058 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.689647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 376/742] 10.222.0.88 () {56 vars in 1124 bytes} [Tue Aug 23 22:10:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:15.691752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 367/743] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:15.692598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bcd4a1-7657-4fc3-8b8b-0607095b0991 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:15.698588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5981d13-90f0-49cb-a4dd-291440c03058 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:15.701262 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:15.848113 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:15.877033 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bcd4a1-7657-4fc3-8b8b-0607095b0991 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:15.887154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5981d13-90f0-49cb-a4dd-291440c03058 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:15.922325 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8154f6-01f7-42ec-a282-dacdc10a8abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:15.923563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8154f6-01f7-42ec-a282-dacdc10a8abe None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.924059 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8154f6-01f7-42ec-a282-dacdc10a8abe None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.924533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8154f6-01f7-42ec-a282-dacdc10a8abe None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.925460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8154f6-01f7-42ec-a282-dacdc10a8abe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:15.925956 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8154f6-01f7-42ec-a282-dacdc10a8abe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:15.928515 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8154f6-01f7-42ec-a282-dacdc10a8abe None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:15.945474 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6c799-dca3-499d-9b48-a0d5f530f78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:15.946224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6c799-dca3-499d-9b48-a0d5f530f78d None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.946568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6c799-dca3-499d-9b48-a0d5f530f78d None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.946904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6c799-dca3-499d-9b48-a0d5f530f78d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.952214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6c799-dca3-499d-9b48-a0d5f530f78d None admin] RBAC: Adding query filter params (interface=public, service_id=19d7ae0a1992409296d9e1feb8745830, region_id=RegionOne) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:15.952214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6c799-dca3-499d-9b48-a0d5f530f78d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:15.952214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6c799-dca3-499d-9b48-a0d5f530f78d None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:15.953819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 377/744] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:15.955872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd6bb9-c27e-4dd9-a6b0-9f529462172a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.956359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd6bb9-c27e-4dd9-a6b0-9f529462172a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.956821 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd6bb9-c27e-4dd9-a6b0-9f529462172a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.963900 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 368/745] 10.222.0.88 () {56 vars in 1251 bytes} [Tue Aug 23 22:10:15 2022] GET /identity/v3/endpoints?service_id=19d7ae0a1992409296d9e1feb8745830&interface=public®ion_id=RegionOne => generated 194 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:15.969102 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6c799-dca3-499d-9b48-a0d5f530f78d None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:15.990696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-34fd6bb9-c27e-4dd9-a6b0-9f529462172a None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:15.992839 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-34fd6bb9-c27e-4dd9-a6b0-9f529462172a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:16.057357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:16.089081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 378/746] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:16.089081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd6bb9-c27e-4dd9-a6b0-9f529462172a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:16.131199 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6c799-dca3-499d-9b48-a0d5f530f78d None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:16.223722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645794c-379f-426e-b44b-8c09481448e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:16.225484 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645794c-379f-426e-b44b-8c09481448e7 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:16.228787 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645794c-379f-426e-b44b-8c09481448e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:16.229338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645794c-379f-426e-b44b-8c09481448e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:16.230334 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645794c-379f-426e-b44b-8c09481448e7 None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=601a3defca70428689ee22517d7bcb19) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:16.230879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645794c-379f-426e-b44b-8c09481448e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:16.232571 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645794c-379f-426e-b44b-8c09481448e7 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:16.249436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:16.252538 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 369/747] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:15 2022] GET /identity/v3/role_assignments?user.id=601a3defca70428689ee22517d7bcb19&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=dc5d71ab685e420697f520aa3a368177 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:16.294539 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd6bb9-c27e-4dd9-a6b0-9f529462172a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:16.360972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f6e53-81f1-40bf-9ed6-8abc0ce7184e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:16.361644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f6e53-81f1-40bf-9ed6-8abc0ce7184e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:16.361927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f6e53-81f1-40bf-9ed6-8abc0ce7184e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:16.362205 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f6e53-81f1-40bf-9ed6-8abc0ce7184e None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:16.362836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f6e53-81f1-40bf-9ed6-8abc0ce7184e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:16.364286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f6e53-81f1-40bf-9ed6-8abc0ce7184e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:16.370949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 379/748] 10.222.0.88 () {56 vars in 1087 bytes} [Tue Aug 23 22:10:16 2022] GET /identity/v3/services => generated 772 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:16.445665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7ba55-d37e-4d53-97bb-ee7ff62a147f None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:16.446190 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7ba55-d37e-4d53-97bb-ee7ff62a147f None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:16.446716 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7ba55-d37e-4d53-97bb-ee7ff62a147f None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:16.449990 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 370/749] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:16.458835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e617126-4ca6-48e3-b2a1-0edbe17889e1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:16.459108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e617126-4ca6-48e3-b2a1-0edbe17889e1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:16.459472 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e617126-4ca6-48e3-b2a1-0edbe17889e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:16.486123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6e617126-4ca6-48e3-b2a1-0edbe17889e1 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:16.487378 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6e617126-4ca6-48e3-b2a1-0edbe17889e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:16.541832 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7ba55-d37e-4d53-97bb-ee7ff62a147f None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:16.598920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 380/750] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:16.716283 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7ba55-d37e-4d53-97bb-ee7ff62a147f None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:16.755192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:16.772855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541cc93-3150-45e4-94b6-2f4533ce634a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:16.773706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541cc93-3150-45e4-94b6-2f4533ce634a None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:16.773910 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541cc93-3150-45e4-94b6-2f4533ce634a None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:16.774158 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541cc93-3150-45e4-94b6-2f4533ce634a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:16.791202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541cc93-3150-45e4-94b6-2f4533ce634a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:16.796737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-4541cc93-3150-45e4-94b6-2f4533ce634a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:16.796737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:16.796737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:16.796737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:16.796737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:16.796737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:16.796737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:16.796737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:16.796737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:16.796737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:16.796737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:16.796737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:16.796737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:16.796737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:16.796737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:16.796737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:16.799700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:16.799700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:16.799700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:16.799700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:16.799700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:16.799700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:16.799700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:16.799700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:16.799700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:16.799700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:16.799700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:16.799700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:16.799700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:16.799700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:16.799700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:16.799700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:16.802324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:16.802324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:16.802324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:16.802324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:16.802324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:16.802324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:16.802324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:16.802324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:16.802324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:16.802324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:16.802324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:16.802324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:16.802324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 371/751] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:16.808604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e617126-4ca6-48e3-b2a1-0edbe17889e1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:16.927942 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:16.997635 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e617126-4ca6-48e3-b2a1-0edbe17889e1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:17.052912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956646c1-8682-4321-88fd-4a03d90ba531 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:17.053287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956646c1-8682-4321-88fd-4a03d90ba531 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:17.053540 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956646c1-8682-4321-88fd-4a03d90ba531 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:17.055282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 372/752] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:17.061670 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d17dc-5bf4-445b-a1f8-55a49f60c480 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:17.063256 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d17dc-5bf4-445b-a1f8-55a49f60c480 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:17.064023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d17dc-5bf4-445b-a1f8-55a49f60c480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:17.069837 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18215d6e-aebc-4adc-a030-ec06661abfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:17.070469 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18215d6e-aebc-4adc-a030-ec06661abfdc None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:17.070750 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18215d6e-aebc-4adc-a030-ec06661abfdc None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:17.071024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18215d6e-aebc-4adc-a030-ec06661abfdc None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:17.071699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18215d6e-aebc-4adc-a030-ec06661abfdc None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:17.072000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18215d6e-aebc-4adc-a030-ec06661abfdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:17.073904 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18215d6e-aebc-4adc-a030-ec06661abfdc None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:17.096869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-753d17dc-5bf4-445b-a1f8-55a49f60c480 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:17.098684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-753d17dc-5bf4-445b-a1f8-55a49f60c480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:17.104329 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 381/753] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:17.111989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18215d6e-aebc-4adc-a030-ec06661abfdc None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:17.236163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 373/754] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:10:17.238265 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d17dc-5bf4-445b-a1f8-55a49f60c480 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:17.361967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18215d6e-aebc-4adc-a030-ec06661abfdc None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:17.429741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:17.457228 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04092d-48da-4b74-9e29-1cb55ab05274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:17.458238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04092d-48da-4b74-9e29-1cb55ab05274 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:17.458527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04092d-48da-4b74-9e29-1cb55ab05274 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:17.458803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04092d-48da-4b74-9e29-1cb55ab05274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:17.476688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d17dc-5bf4-445b-a1f8-55a49f60c480 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:17.489836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04092d-48da-4b74-9e29-1cb55ab05274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:17.491407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04092d-48da-4b74-9e29-1cb55ab05274 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:17.514863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-2c04092d-48da-4b74-9e29-1cb55ab05274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:17.514863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:17.514863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:17.514863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:17.514863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:17.514863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:17.514863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:17.514863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:17.514863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:17.514863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:17.514863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:17.514863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:17.514863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:17.514863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:17.514863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:17.514863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:17.517281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:17.519120 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:17.519120 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:17.519120 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:17.519120 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 382/755] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:17.529443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:17.531847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04092d-48da-4b74-9e29-1cb55ab05274 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:17.556769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e1d1c-03a7-4a74-a44d-330ee093c66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:17.557837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e1d1c-03a7-4a74-a44d-330ee093c66c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:17.557837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e1d1c-03a7-4a74-a44d-330ee093c66c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:17.558340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e1d1c-03a7-4a74-a44d-330ee093c66c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:17.589129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e1d1c-03a7-4a74-a44d-330ee093c66c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:17.594156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-eb8e1d1c-03a7-4a74-a44d-330ee093c66c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:17.594156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:17.594156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:17.594156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:17.594156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:17.594156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:17.594156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:17.594156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:17.594156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:17.594156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:17.594156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:17.594156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:17.594156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:17.594156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:17.594156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:17.594156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:17.595987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:17.595987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:17.595987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:17.595987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:17.595987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:17.595987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:17.595987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:17.595987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:17.595987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:17.595987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:17.595987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:17.595987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:17.595987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:17.595987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:17.595987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:17.595987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:17.597764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:17.597764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:17.597764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:17.597764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:17.597764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:17.597764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:17.597764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:17.597764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:17.597764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:17.597764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:17.597764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:17.597764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:17.597764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 374/756] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:17.609448 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e1d1c-03a7-4a74-a44d-330ee093c66c None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:17.731814 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04092d-48da-4b74-9e29-1cb55ab05274 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:17.781889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e1d1c-03a7-4a74-a44d-330ee093c66c None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:17.784762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b95cf-899b-4edb-be65-6364a36cf24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:17.786487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b95cf-899b-4edb-be65-6364a36cf24b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:17.786901 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b95cf-899b-4edb-be65-6364a36cf24b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:17.787273 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b95cf-899b-4edb-be65-6364a36cf24b None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:17.788025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b95cf-899b-4edb-be65-6364a36cf24b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:17.788427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b95cf-899b-4edb-be65-6364a36cf24b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:17.789925 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b95cf-899b-4edb-be65-6364a36cf24b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:17.809100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 383/757] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:17.810192 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b95cf-899b-4edb-be65-6364a36cf24b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:17.829984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b2f5e-83f4-446f-947d-d032031f1700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:17.830909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b2f5e-83f4-446f-947d-d032031f1700 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:17.831362 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b2f5e-83f4-446f-947d-d032031f1700 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:17.831781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b2f5e-83f4-446f-947d-d032031f1700 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:17.832629 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b2f5e-83f4-446f-947d-d032031f1700 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:17.833111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b2f5e-83f4-446f-947d-d032031f1700 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:17.834634 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b2f5e-83f4-446f-947d-d032031f1700 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:17.844578 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 375/758] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:17.845756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b2f5e-83f4-446f-947d-d032031f1700 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:17.869027 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:17.897416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:17.934156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b95cf-899b-4edb-be65-6364a36cf24b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:17.963909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b2f5e-83f4-446f-947d-d032031f1700 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:17.971932 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001b8b1-1c60-4c3c-8c7e-2a0e3fe87bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:17.972569 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001b8b1-1c60-4c3c-8c7e-2a0e3fe87bec None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:17.972846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001b8b1-1c60-4c3c-8c7e-2a0e3fe87bec None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:17.973157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001b8b1-1c60-4c3c-8c7e-2a0e3fe87bec None admin] PATH_INFO: `/v3/services/network` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:17.974109 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3001b8b1-1c60-4c3c-8c7e-2a0e3fe87bec None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:17.975104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3001b8b1-1c60-4c3c-8c7e-2a0e3fe87bec None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-3001b8b1-1c60-4c3c-8c7e-2a0e3fe87bec None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:17.984551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:17.985572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:17.985572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:17.985572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:17.985572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:17.985572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:17.985572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:17.985572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:17.985572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:17.985572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:17.985572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:17.985572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:17.985572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:17.985572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:17.985572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:17.985572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:17.985572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:17.986462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:17.986462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:17.986462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:17.986462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:17.986462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:17.986462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:17.986462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:17.986462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:17.986462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:17.986462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:17.986462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:17.986462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:17.986462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:17.986462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:17.990967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 384/759] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:17 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:17.991893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001b8b1-1c60-4c3c-8c7e-2a0e3fe87bec None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:18.008885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac6a92-8b03-4750-ab60-051c280dfd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:18.009937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac6a92-8b03-4750-ab60-051c280dfd3e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:18.010422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac6a92-8b03-4750-ab60-051c280dfd3e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:18.010933 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac6a92-8b03-4750-ab60-051c280dfd3e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:18.021151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac6a92-8b03-4750-ab60-051c280dfd3e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:18.022986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac6a92-8b03-4750-ab60-051c280dfd3e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:18.035930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-39ac6a92-8b03-4750-ab60-051c280dfd3e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:10:18.035930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:18.035930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:18.035930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:18.035930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:18.035930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:18.035930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:18.035930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:18.035930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:18.035930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:18.035930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:18.035930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:18.035930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:18.035930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:18.035930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:18.035930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:18.037024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:18.038356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:18.038356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:18.038356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:18.038356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:18.038356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:18.038356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:18.038356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:18.038356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:18.038356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:10:18.038356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:18.040557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 376/760] 10.222.0.88 () {56 vars in 1117 bytes} [Tue Aug 23 22:10:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:18.041649 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac6a92-8b03-4750-ab60-051c280dfd3e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:18.102972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001b8b1-1c60-4c3c-8c7e-2a0e3fe87bec None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:18.105668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:18.141882 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bfcb60-3ac6-4897-b42c-7a6c89cfd65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:18.142518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bfcb60-3ac6-4897-b42c-7a6c89cfd65e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:18.142795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bfcb60-3ac6-4897-b42c-7a6c89cfd65e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:18.143061 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bfcb60-3ac6-4897-b42c-7a6c89cfd65e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:18.153837 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bfcb60-3ac6-4897-b42c-7a6c89cfd65e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:18.155421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bfcb60-3ac6-4897-b42c-7a6c89cfd65e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:18.163668 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac6a92-8b03-4750-ab60-051c280dfd3e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:18.165713 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:18.173632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-88bfcb60-3ac6-4897-b42c-7a6c89cfd65e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:18.173632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:18.173632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:18.173632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:18.173632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:18.173632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:18.173632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:18.173632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:18.173632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:18.173632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:18.173632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:18.173632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:18.173632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:18.173632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:18.173632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:18.173632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:18.174691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:18.175647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:18.175647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:18.175647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:18.175647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 385/761] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:18.176303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bfcb60-3ac6-4897-b42c-7a6c89cfd65e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:18.214734 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914095b2-e086-4959-b91d-f5f41df87736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:18.215756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914095b2-e086-4959-b91d-f5f41df87736 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:18.216310 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914095b2-e086-4959-b91d-f5f41df87736 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:18.216886 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914095b2-e086-4959-b91d-f5f41df87736 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:18.217883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914095b2-e086-4959-b91d-f5f41df87736 None admin] RBAC: Adding query filter params (name=network) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:18.218998 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914095b2-e086-4959-b91d-f5f41df87736 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:18.221131 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914095b2-e086-4959-b91d-f5f41df87736 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:18.227655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 377/762] 10.222.0.88 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:18.228819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914095b2-e086-4959-b91d-f5f41df87736 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:18.297266 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bfcb60-3ac6-4897-b42c-7a6c89cfd65e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:18.350203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c932a47-f86a-4b4c-88a0-7cdadc5702ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:18.351258 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c932a47-f86a-4b4c-88a0-7cdadc5702ae None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:18.351758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c932a47-f86a-4b4c-88a0-7cdadc5702ae None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:18.352248 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c932a47-f86a-4b4c-88a0-7cdadc5702ae None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:18.353622 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c932a47-f86a-4b4c-88a0-7cdadc5702ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:18.353939 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c932a47-f86a-4b4c-88a0-7cdadc5702ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:18.356616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c932a47-f86a-4b4c-88a0-7cdadc5702ae None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:18.393500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 386/763] 10.222.0.88 () {56 vars in 1149 bytes} [Tue Aug 23 22:10:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:18.395124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c932a47-f86a-4b4c-88a0-7cdadc5702ae None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:18.425961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:18.496915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914095b2-e086-4959-b91d-f5f41df87736 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:18.531309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c932a47-f86a-4b4c-88a0-7cdadc5702ae None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:18.539316 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06088810-946e-439a-973a-c11c9e6077f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:18.540427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06088810-946e-439a-973a-c11c9e6077f0 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:18.540940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06088810-946e-439a-973a-c11c9e6077f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:18.541469 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06088810-946e-439a-973a-c11c9e6077f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:18.542433 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06088810-946e-439a-973a-c11c9e6077f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:18.542938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06088810-946e-439a-973a-c11c9e6077f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:18.544707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06088810-946e-439a-973a-c11c9e6077f0 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:18.560575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 378/764] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:18.563580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06088810-946e-439a-973a-c11c9e6077f0 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:18.572968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f45a0-afe5-42a2-8286-655dfd057aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:18.573613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f45a0-afe5-42a2-8286-655dfd057aff None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:18.573894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f45a0-afe5-42a2-8286-655dfd057aff None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:18.574164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f45a0-afe5-42a2-8286-655dfd057aff None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:18.574799 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f45a0-afe5-42a2-8286-655dfd057aff None admin] RBAC: Adding query filter params (type=network) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:18.575101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f45a0-afe5-42a2-8286-655dfd057aff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:18.576473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f45a0-afe5-42a2-8286-655dfd057aff None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:18.587710 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 387/765] 10.222.0.88 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:18.588652 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f45a0-afe5-42a2-8286-655dfd057aff None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:18.601170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:18.679603 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:18.695215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06088810-946e-439a-973a-c11c9e6077f0 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:18.705720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f45a0-afe5-42a2-8286-655dfd057aff None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:18.737367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647cf00d-a691-4e09-80cd-0be2f7cdbf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:18.738505 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647cf00d-a691-4e09-80cd-0be2f7cdbf05 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:18.739088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647cf00d-a691-4e09-80cd-0be2f7cdbf05 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:18.739567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647cf00d-a691-4e09-80cd-0be2f7cdbf05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:18.744826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2d42b-09dd-4b88-8dc7-83366706ace6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:18.744826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2d42b-09dd-4b88-8dc7-83366706ace6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:18.745140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2d42b-09dd-4b88-8dc7-83366706ace6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:18.745419 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2d42b-09dd-4b88-8dc7-83366706ace6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:18.754522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2d42b-09dd-4b88-8dc7-83366706ace6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:18.756423 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2d42b-09dd-4b88-8dc7-83366706ace6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:18.775407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-e2c2d42b-09dd-4b88-8dc7-83366706ace6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:18.775407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:18.775407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:18.775407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:18.775407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:18.775407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:18.775407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:18.775407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:18.775407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:18.775407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:18.775407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:18.775407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:18.775407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:18.775407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:18.775407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:18.775407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:18.776487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:18.778156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:18.778156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:18.778156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:18.778156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:18.778156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:18.778156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:18.778156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:18.778156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:18.778156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:18.778156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:18.778156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 388/766] 10.222.0.88 () {56 vars in 1093 bytes} [Tue Aug 23 22:10:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:18.778825 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2d42b-09dd-4b88-8dc7-83366706ace6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:18.839989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647cf00d-a691-4e09-80cd-0be2f7cdbf05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:18.841437 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647cf00d-a691-4e09-80cd-0be2f7cdbf05 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:18.873016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:18.966226 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 379/767] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:10:18.966226 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647cf00d-a691-4e09-80cd-0be2f7cdbf05 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:18.971178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2d42b-09dd-4b88-8dc7-83366706ace6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:19.026053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286960c-d8ff-481e-8f74-848015b465cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:19.026917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286960c-d8ff-481e-8f74-848015b465cc None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.027383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286960c-d8ff-481e-8f74-848015b465cc None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.027711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286960c-d8ff-481e-8f74-848015b465cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.028462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286960c-d8ff-481e-8f74-848015b465cc None admin] RBAC: Adding query filter params (interface=public, service_id=b37aa296901a42acb4976ebe0eca37d8, region_id=RegionOne) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:19.028998 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286960c-d8ff-481e-8f74-848015b465cc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:19.031935 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286960c-d8ff-481e-8f74-848015b465cc None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:19.043914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 389/768] 10.222.0.88 () {56 vars in 1251 bytes} [Tue Aug 23 22:10:18 2022] GET /identity/v3/endpoints?service_id=b37aa296901a42acb4976ebe0eca37d8&interface=public®ion_id=RegionOne => generated 194 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:19.045374 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17ed29-c6ff-4bd2-a9b7-f5b7bf7cbf38 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.045667 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17ed29-c6ff-4bd2-a9b7-f5b7bf7cbf38 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.045985 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17ed29-c6ff-4bd2-a9b7-f5b7bf7cbf38 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.048773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 390/769] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:19.054747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17ed29-c6ff-4bd2-a9b7-f5b7bf7cbf38 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:19.058407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:19.117372 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647cf00d-a691-4e09-80cd-0be2f7cdbf05 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:19.149208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:19.162089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5b695-aa8b-400e-a29e-063bcb8ecf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:19.162891 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5b695-aa8b-400e-a29e-063bcb8ecf7b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.163304 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5b695-aa8b-400e-a29e-063bcb8ecf7b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.163769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5b695-aa8b-400e-a29e-063bcb8ecf7b None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.164756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5b695-aa8b-400e-a29e-063bcb8ecf7b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:19.165080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5b695-aa8b-400e-a29e-063bcb8ecf7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:19.166779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5b695-aa8b-400e-a29e-063bcb8ecf7b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:19.193106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 380/770] 10.222.0.88 () {56 vars in 1133 bytes} [Tue Aug 23 22:10:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:19.193106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5b695-aa8b-400e-a29e-063bcb8ecf7b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:19.200684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17ed29-c6ff-4bd2-a9b7-f5b7bf7cbf38 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:19.244720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75478324-9812-447c-968c-bd703cef46a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:19.245419 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75478324-9812-447c-968c-bd703cef46a2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.245768 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75478324-9812-447c-968c-bd703cef46a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.246147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75478324-9812-447c-968c-bd703cef46a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.258861 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75478324-9812-447c-968c-bd703cef46a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:19.260359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75478324-9812-447c-968c-bd703cef46a2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:19.277541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-75478324-9812-447c-968c-bd703cef46a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:19.277541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:19.277541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:19.277541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:19.277541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:19.277541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:19.277541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:19.277541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:19.277541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:19.277541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:19.277541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:19.277541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:19.277541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:19.277541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:19.277541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:19.277541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:19.278647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:19.280277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:19.280277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:19.280277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:19.280277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 391/771] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:19.280277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b55bd9-2d1c-4267-9477-27288b6c5f4c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.280885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b55bd9-2d1c-4267-9477-27288b6c5f4c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.280885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b55bd9-2d1c-4267-9477-27288b6c5f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.302370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-36b55bd9-2d1c-4267-9477-27288b6c5f4c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:19.303877 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-36b55bd9-2d1c-4267-9477-27288b6c5f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:19.321676 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5b695-aa8b-400e-a29e-063bcb8ecf7b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:19.359808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb25c1e-c7b8-48ea-ace7-e1cb806f3711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:19.360503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb25c1e-c7b8-48ea-ace7-e1cb806f3711 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.360865 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb25c1e-c7b8-48ea-ace7-e1cb806f3711 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.361245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb25c1e-c7b8-48ea-ace7-e1cb806f3711 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.362033 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb25c1e-c7b8-48ea-ace7-e1cb806f3711 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:19.362223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:19.364050 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb25c1e-c7b8-48ea-ace7-e1cb806f3711 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:19.370555 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 381/772] 10.222.0.88 () {56 vars in 1087 bytes} [Tue Aug 23 22:10:19 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:19.371539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb25c1e-c7b8-48ea-ace7-e1cb806f3711 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:19.394261 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 392/773] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:19.395872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b55bd9-2d1c-4267-9477-27288b6c5f4c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:19.461652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:19.561302 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb25c1e-c7b8-48ea-ace7-e1cb806f3711 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:19.566528 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b55bd9-2d1c-4267-9477-27288b6c5f4c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:19.613912 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d48a1-e106-4283-961a-0ff19a12e53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:19.614540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d48a1-e106-4283-961a-0ff19a12e53e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.614814 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d48a1-e106-4283-961a-0ff19a12e53e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.615143 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d48a1-e106-4283-961a-0ff19a12e53e None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.615790 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d48a1-e106-4283-961a-0ff19a12e53e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:19.616140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d48a1-e106-4283-961a-0ff19a12e53e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:19.617578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d48a1-e106-4283-961a-0ff19a12e53e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:19.621426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61176708-e812-4d07-9026-afe11208d8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:19.622740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61176708-e812-4d07-9026-afe11208d8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.623406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61176708-e812-4d07-9026-afe11208d8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.624059 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61176708-e812-4d07-9026-afe11208d8b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.631674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 393/774] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:19.632943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecc2b0-0eb6-4102-b8bf-877189c9cd9b None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.637749 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecc2b0-0eb6-4102-b8bf-877189c9cd9b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.638067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecc2b0-0eb6-4102-b8bf-877189c9cd9b None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.639704 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 394/775] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:19.646861 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ecc2b0-0eb6-4102-b8bf-877189c9cd9b None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:19.652005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61176708-e812-4d07-9026-afe11208d8b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:19.654870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61176708-e812-4d07-9026-afe11208d8b4 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:19.683015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-61176708-e812-4d07-9026-afe11208d8b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:19.683015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:19.683015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:19.683015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:19.683015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:19.683015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:19.683015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:19.683015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:19.683015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:19.683015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:19.683015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:19.683015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:19.683015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:19.683015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:19.683015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:19.683015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:19.684726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:19.686390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:19.686390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:19.686390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:19.690214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 382/776] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:19.693014 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c554e0a-97c7-4107-b755-d9fc07a6c70d None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.693739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c554e0a-97c7-4107-b755-d9fc07a6c70d None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.694407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c554e0a-97c7-4107-b755-d9fc07a6c70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.704441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:19.727950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-6c554e0a-97c7-4107-b755-d9fc07a6c70d None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:19.730614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6c554e0a-97c7-4107-b755-d9fc07a6c70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:19.776930 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ecc2b0-0eb6-4102-b8bf-877189c9cd9b None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:19.824307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa84bfb-c09c-4fd6-8865-5cc508ad779f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:19.825373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa84bfb-c09c-4fd6-8865-5cc508ad779f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.826467 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa84bfb-c09c-4fd6-8865-5cc508ad779f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.826966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa84bfb-c09c-4fd6-8865-5cc508ad779f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.836123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa84bfb-c09c-4fd6-8865-5cc508ad779f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:19.838563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa84bfb-c09c-4fd6-8865-5cc508ad779f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:19.854152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-9aa84bfb-c09c-4fd6-8865-5cc508ad779f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:19.854152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:19.854152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:19.854152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:19.854152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:19.854152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:19.854152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:19.854152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:19.854152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:19.854152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:19.854152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:19.854152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:19.854152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:19.854152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:19.854152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:19.855093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:19.855968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:19.855968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:19.855968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:19.855968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:19.855968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:19.855968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:19.855968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:19.855968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:19.858116 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 395/777] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:19.859698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa84bfb-c09c-4fd6-8865-5cc508ad779f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:19.862363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 383/778] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:19.875205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c554e0a-97c7-4107-b755-d9fc07a6c70d None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:19.923720 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:20.015664 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c554e0a-97c7-4107-b755-d9fc07a6c70d None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:20.033822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa84bfb-c09c-4fd6-8865-5cc508ad779f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:20.037186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:20.063435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21678f7a-d1dd-4ff0-9c3d-d304cac85c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:20.064309 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21678f7a-d1dd-4ff0-9c3d-d304cac85c15 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.064850 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21678f7a-d1dd-4ff0-9c3d-d304cac85c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.065293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21678f7a-d1dd-4ff0-9c3d-d304cac85c15 None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.066254 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21678f7a-d1dd-4ff0-9c3d-d304cac85c15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:20.066814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21678f7a-d1dd-4ff0-9c3d-d304cac85c15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:20.068559 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21678f7a-d1dd-4ff0-9c3d-d304cac85c15 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:20.078488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924373b-d500-4645-87f6-164d403dcdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:20.079766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924373b-d500-4645-87f6-164d403dcdd7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.079766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924373b-d500-4645-87f6-164d403dcdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.080238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924373b-d500-4645-87f6-164d403dcdd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.081084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924373b-d500-4645-87f6-164d403dcdd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:20.081479 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924373b-d500-4645-87f6-164d403dcdd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:20.082964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924373b-d500-4645-87f6-164d403dcdd7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:20.083995 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 384/779] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:20.085187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21678f7a-d1dd-4ff0-9c3d-d304cac85c15 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:20.102569 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 396/780] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:20.103796 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924373b-d500-4645-87f6-164d403dcdd7 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:20.191530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:20.204593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21678f7a-d1dd-4ff0-9c3d-d304cac85c15 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:20.220565 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924373b-d500-4645-87f6-164d403dcdd7 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:20.244074 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6d827-a34e-461f-9247-60655d22cf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:20.245212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6d827-a34e-461f-9247-60655d22cf92 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.245710 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6d827-a34e-461f-9247-60655d22cf92 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.246302 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6d827-a34e-461f-9247-60655d22cf92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.257169 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47cb0c1-fb35-46f3-a1ba-272e8050fcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:20.257789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cb0c1-fb35-46f3-a1ba-272e8050fcec None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.258070 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cb0c1-fb35-46f3-a1ba-272e8050fcec None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.258344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cb0c1-fb35-46f3-a1ba-272e8050fcec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.259104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47cb0c1-fb35-46f3-a1ba-272e8050fcec None admin] RBAC: Adding query filter params (role.id=23ea214daa734346a6da3a0a8c50d23a, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=cb001b14eec54047aee42ca6bd4a1eba) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:20.259419 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47cb0c1-fb35-46f3-a1ba-272e8050fcec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:20.259938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6d827-a34e-461f-9247-60655d22cf92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:20.260886 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47cb0c1-fb35-46f3-a1ba-272e8050fcec None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:20.267014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 397/781] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:20 2022] GET /identity/v3/role_assignments?user.id=cb001b14eec54047aee42ca6bd4a1eba&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=23ea214daa734346a6da3a0a8c50d23a => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:20.267014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47cb0c1-fb35-46f3-a1ba-272e8050fcec None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:20.270637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-5de6d827-a34e-461f-9247-60655d22cf92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:20.270637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:20.270637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:20.270637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:20.270637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:20.270637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:20.270637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:20.270637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:20.270637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:20.270637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:20.270637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:20.270637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:20.270637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:20.270637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:20.270637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:20.270637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:20.271688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:20.271688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:20.271688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:20.271688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:20.271688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:20.271688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:20.271688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:20.271688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:20.271688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:20.271688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:20.271688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:20.271688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:20.271688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:20.271688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:20.271688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:20.271688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:20.272550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:20.272550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:20.272550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:20.272550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:20.272550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:20.272550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:20.272550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:20.272550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:20.272550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:20.272550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:20.272550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:20.272550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:20.277219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 385/782] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:20.278398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6d827-a34e-461f-9247-60655d22cf92 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:20.313073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:20.431722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6d827-a34e-461f-9247-60655d22cf92 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:20.433366 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47cb0c1-fb35-46f3-a1ba-272e8050fcec None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:20.490208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e67110-af85-479f-97ca-8254b6dda8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:20.490760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67110-af85-479f-97ca-8254b6dda8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.491031 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67110-af85-479f-97ca-8254b6dda8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.491306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67110-af85-479f-97ca-8254b6dda8b1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.491903 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e67110-af85-479f-97ca-8254b6dda8b1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:20.493345 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e67110-af85-479f-97ca-8254b6dda8b1 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:20.503467 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e95046-e8e4-4b19-8914-f8e7fb45c465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-b7e67110-af85-479f-97ca-8254b6dda8b1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:20.503997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:20.505842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:20.505842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:20.505842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:20.505842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:20.505842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:20.505842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:20.505842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:20.505842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:20.505842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:20.505842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:20.505842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:20.505842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:20.505842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:20.505842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:20.505842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:20.505842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:20.507500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:20.507500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:20.507500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:20.507500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:20.507500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:20.507500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:20.507500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:20.507500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:20.507500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:20.507500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:20.507500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:20.507500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:20.507500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:20.507500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:20.509801 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 386/783] 10.222.0.88 () {56 vars in 1129 bytes} [Tue Aug 23 22:10:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:20.510293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e67110-af85-479f-97ca-8254b6dda8b1 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:20.511759 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e95046-e8e4-4b19-8914-f8e7fb45c465 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.512561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e95046-e8e4-4b19-8914-f8e7fb45c465 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.513370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e95046-e8e4-4b19-8914-f8e7fb45c465 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.533356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e95046-e8e4-4b19-8914-f8e7fb45c465 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:20.534006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:20.535152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e95046-e8e4-4b19-8914-f8e7fb45c465 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:20.549677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-f1e95046-e8e4-4b19-8914-f8e7fb45c465 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:20.549677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:20.549677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:20.549677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:20.549677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:20.549677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:20.549677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:20.549677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:20.549677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:20.549677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:20.549677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:20.549677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:20.549677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:20.549677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:20.549677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:20.551351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:20.553138 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:20.553138 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:20.553138 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:20.553138 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:20.553138 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:20.553138 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:20.553138 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:20.553138 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:20.553138 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 398/784] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:20.553138 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e95046-e8e4-4b19-8914-f8e7fb45c465 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:20.646711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e67110-af85-479f-97ca-8254b6dda8b1 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:20.685190 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f70ea-d251-40bf-b5ae-5f05fea2e2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:20.685903 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f70ea-d251-40bf-b5ae-5f05fea2e2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.686185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f70ea-d251-40bf-b5ae-5f05fea2e2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.686462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f70ea-d251-40bf-b5ae-5f05fea2e2c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.687945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f70ea-d251-40bf-b5ae-5f05fea2e2c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:20.688250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f70ea-d251-40bf-b5ae-5f05fea2e2c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:20.689950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f70ea-d251-40bf-b5ae-5f05fea2e2c6 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:20.701076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 387/785] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:20.702053 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f70ea-d251-40bf-b5ae-5f05fea2e2c6 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:20.748827 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:20.778265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e95046-e8e4-4b19-8914-f8e7fb45c465 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:20.808347 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:20.848120 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f020e-5e21-409f-a87d-d71f0e498c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:20.848845 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f020e-5e21-409f-a87d-d71f0e498c20 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.849312 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f020e-5e21-409f-a87d-d71f0e498c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.849657 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f020e-5e21-409f-a87d-d71f0e498c20 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.850389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f020e-5e21-409f-a87d-d71f0e498c20 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:20.850814 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f020e-5e21-409f-a87d-d71f0e498c20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:20.851999 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f70ea-d251-40bf-b5ae-5f05fea2e2c6 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:20.852769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f020e-5e21-409f-a87d-d71f0e498c20 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:20.871708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 399/786] 10.222.0.88 () {56 vars in 1124 bytes} [Tue Aug 23 22:10:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:20.871708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f020e-5e21-409f-a87d-d71f0e498c20 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:20.942858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc86e7-092a-447f-9bd8-ee4a92539f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:20.945363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc86e7-092a-447f-9bd8-ee4a92539f68 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.946301 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc86e7-092a-447f-9bd8-ee4a92539f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.948102 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc86e7-092a-447f-9bd8-ee4a92539f68 None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.949958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc86e7-092a-447f-9bd8-ee4a92539f68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:20.951171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc86e7-092a-447f-9bd8-ee4a92539f68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:20.955892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc86e7-092a-447f-9bd8-ee4a92539f68 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:20.987268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 388/787] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:20.989579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc86e7-092a-447f-9bd8-ee4a92539f68 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:21.039620 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f020e-5e21-409f-a87d-d71f0e498c20 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:21.103169 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4286fe-3821-4b4b-8b55-c034b9539963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:21.104842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4286fe-3821-4b4b-8b55-c034b9539963 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.105633 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4286fe-3821-4b4b-8b55-c034b9539963 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.106477 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4286fe-3821-4b4b-8b55-c034b9539963 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.128396 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4286fe-3821-4b4b-8b55-c034b9539963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:21.130160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc86e7-092a-447f-9bd8-ee4a92539f68 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:21.130555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4286fe-3821-4b4b-8b55-c034b9539963 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:21.149684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-9a4286fe-3821-4b4b-8b55-c034b9539963 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:21.149684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:21.149684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:21.149684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:21.149684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:21.149684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:21.149684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:21.149684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:21.149684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:21.149684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:21.149684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:21.149684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:21.149684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:21.149684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:21.149684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:21.149684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:21.150785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:21.151765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:21.151765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:21.151765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:21.154090 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 400/788] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:21.155100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4286fe-3821-4b4b-8b55-c034b9539963 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:21.159826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:21.170567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d0daa-67fe-4b11-be13-acd3c49c467d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:21.171420 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d0daa-67fe-4b11-be13-acd3c49c467d None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.171753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d0daa-67fe-4b11-be13-acd3c49c467d None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.172094 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d0daa-67fe-4b11-be13-acd3c49c467d None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.172825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d0daa-67fe-4b11-be13-acd3c49c467d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:21.173386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d0daa-67fe-4b11-be13-acd3c49c467d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:21.175140 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d0daa-67fe-4b11-be13-acd3c49c467d None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:21.177631 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:21.187962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 389/789] 10.222.0.88 () {56 vars in 1124 bytes} [Tue Aug 23 22:10:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:21.189158 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d0daa-67fe-4b11-be13-acd3c49c467d None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:21.312741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d0daa-67fe-4b11-be13-acd3c49c467d None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:21.355895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78812c-7147-498e-a891-c4f1ac0c4e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:21.356683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78812c-7147-498e-a891-c4f1ac0c4e22 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.357161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78812c-7147-498e-a891-c4f1ac0c4e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.357546 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78812c-7147-498e-a891-c4f1ac0c4e22 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.358304 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78812c-7147-498e-a891-c4f1ac0c4e22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:21.359387 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78812c-7147-498e-a891-c4f1ac0c4e22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:21.360898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78812c-7147-498e-a891-c4f1ac0c4e22 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:21.374181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 390/790] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:21.375211 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78812c-7147-498e-a891-c4f1ac0c4e22 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:21.411788 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4286fe-3821-4b4b-8b55-c034b9539963 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:21.451386 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37540895-ca85-4a34-a9b2-c245083ce211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:21.452527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37540895-ca85-4a34-a9b2-c245083ce211 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.453265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37540895-ca85-4a34-a9b2-c245083ce211 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.453854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37540895-ca85-4a34-a9b2-c245083ce211 None admin] PATH_INFO: `/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/e3517b3b077a4a1fa76163657b46930c/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.456093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37540895-ca85-4a34-a9b2-c245083ce211 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a7317af325f485e8fae49631cb5bc20, user_id=e3517b3b077a4a1fa76163657b46930c, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:21.457884 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37540895-ca85-4a34-a9b2-c245083ce211 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:21.476761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 401/791] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:10:21 2022] PUT /identity/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/e3517b3b077a4a1fa76163657b46930c/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:10:21.478152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37540895-ca85-4a34-a9b2-c245083ce211 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:21.506050 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:21.606705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78812c-7147-498e-a891-c4f1ac0c4e22 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:21.639643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:21.663504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589606e9-20fa-4910-99cf-67ebb349cc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:21.665653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589606e9-20fa-4910-99cf-67ebb349cc2a None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.666233 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589606e9-20fa-4910-99cf-67ebb349cc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.666980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589606e9-20fa-4910-99cf-67ebb349cc2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.668050 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589606e9-20fa-4910-99cf-67ebb349cc2a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:21.670344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589606e9-20fa-4910-99cf-67ebb349cc2a None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:21.685863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37540895-ca85-4a34-a9b2-c245083ce211 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:21.720354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 391/792] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:10:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:21.740075 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a573e84-dcbc-4125-ab81-9909e8ba58cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:21.740769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a573e84-dcbc-4125-ab81-9909e8ba58cd None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.741044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a573e84-dcbc-4125-ab81-9909e8ba58cd None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.741446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a573e84-dcbc-4125-ab81-9909e8ba58cd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.752064 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a573e84-dcbc-4125-ab81-9909e8ba58cd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:21.754668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a573e84-dcbc-4125-ab81-9909e8ba58cd None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:21.771324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-9a573e84-dcbc-4125-ab81-9909e8ba58cd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:21.771324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:21.771324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:21.771324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:21.771324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:21.771324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:21.771324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:21.771324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:21.771324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:21.771324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:21.771324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:21.771324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:21.771324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:21.771324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:21.771324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:21.771324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:21.773078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:21.775764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:21.775764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:21.775764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:21.775764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:21.775764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:21.775764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:21.775764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:21.775764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:21.775764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:21.775764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:21.775764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 402/793] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:21.780010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589606e9-20fa-4910-99cf-67ebb349cc2a None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:21.844370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b3513-ba99-498c-9f50-358790aed717 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.844874 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b3513-ba99-498c-9f50-358790aed717 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.845402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b3513-ba99-498c-9f50-358790aed717 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.850076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 403/794] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:21.854044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a68c67-b63b-4a71-8fd0-eb52fdc27888 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.854327 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a68c67-b63b-4a71-8fd0-eb52fdc27888 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.854602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a68c67-b63b-4a71-8fd0-eb52fdc27888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.893173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-63a68c67-b63b-4a71-8fd0-eb52fdc27888 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:21.894402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-63a68c67-b63b-4a71-8fd0-eb52fdc27888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:21.959300 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589606e9-20fa-4910-99cf-67ebb349cc2a None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:21.990066 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:22.026508 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e770a8e-baf3-4d11-8e40-3d48dfdc9e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:22.028176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 404/795] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:22.028548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e770a8e-baf3-4d11-8e40-3d48dfdc9e1f None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:22.029230 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e770a8e-baf3-4d11-8e40-3d48dfdc9e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:22.029734 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e770a8e-baf3-4d11-8e40-3d48dfdc9e1f None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:22.031464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e770a8e-baf3-4d11-8e40-3d48dfdc9e1f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:22.032005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e770a8e-baf3-4d11-8e40-3d48dfdc9e1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:22.033746 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e770a8e-baf3-4d11-8e40-3d48dfdc9e1f None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:22.054327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 392/796] 10.222.0.88 () {56 vars in 1135 bytes} [Tue Aug 23 22:10:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:22.063963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a68c67-b63b-4a71-8fd0-eb52fdc27888 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:22.125451 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:22.186101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a68c67-b63b-4a71-8fd0-eb52fdc27888 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:22.224416 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cb276-b7e3-429e-ab1c-02c8785eaa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:22.225350 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cb276-b7e3-429e-ab1c-02c8785eaa53 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:22.225805 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cb276-b7e3-429e-ab1c-02c8785eaa53 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:22.226237 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cb276-b7e3-429e-ab1c-02c8785eaa53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:22.239238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0cb276-b7e3-429e-ab1c-02c8785eaa53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:22.240990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0cb276-b7e3-429e-ab1c-02c8785eaa53 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:22.259746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-bb0cb276-b7e3-429e-ab1c-02c8785eaa53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:22.259746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:22.259746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:22.259746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:22.259746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:22.259746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:22.259746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:22.259746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:22.259746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:22.259746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:22.259746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:22.259746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:22.259746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:22.259746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:22.259746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:22.259746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:22.261399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:22.262700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:22.262700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:22.262700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:22.264744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 405/797] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:22.270542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e770a8e-baf3-4d11-8e40-3d48dfdc9e1f None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:22.379197 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cb276-b7e3-429e-ab1c-02c8785eaa53 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:22.486816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e770a8e-baf3-4d11-8e40-3d48dfdc9e1f None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:22.597553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b281554-ba01-4842-bf8b-ea4716adf71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:22.599020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b281554-ba01-4842-bf8b-ea4716adf71c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:22.599709 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b281554-ba01-4842-bf8b-ea4716adf71c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:22.600404 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b281554-ba01-4842-bf8b-ea4716adf71c None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:22.601830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b281554-ba01-4842-bf8b-ea4716adf71c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:22.602577 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b281554-ba01-4842-bf8b-ea4716adf71c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:22.605610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b281554-ba01-4842-bf8b-ea4716adf71c None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:22.628930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 393/798] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:22.638252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b281554-ba01-4842-bf8b-ea4716adf71c None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:22.665347 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cb276-b7e3-429e-ab1c-02c8785eaa53 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:22.668306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:22.716747 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:22.746543 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d78de1-573c-4b16-9b41-ed42e80d4578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:22.747671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d78de1-573c-4b16-9b41-ed42e80d4578 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:22.748323 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d78de1-573c-4b16-9b41-ed42e80d4578 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:22.748796 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d78de1-573c-4b16-9b41-ed42e80d4578 None admin] PATH_INFO: `/v3/services/network` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:22.750427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d78de1-573c-4b16-9b41-ed42e80d4578 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:22.753456 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d78de1-573c-4b16-9b41-ed42e80d4578 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-b5d78de1-573c-4b16-9b41-ed42e80d4578 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:22.773145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:22.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:22.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:22.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:22.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:22.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:22.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:22.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:22.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:22.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:22.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:22.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:22.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:22.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:22.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:22.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:22.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:22.776766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:22.776766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:22.776766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:22.776766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:22.776766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:22.776766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:22.776766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:22.776766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:22.776766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:22.776766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:22.776766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:22.776766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:22.776766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:22.776766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:22.782656 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 406/799] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:22 2022] GET /identity/v3/services/network => generated 88 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:22.785614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba5aec-7ef2-48ee-9e0a-2e03f2af8d44 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:22.787680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba5aec-7ef2-48ee-9e0a-2e03f2af8d44 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:22.788200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba5aec-7ef2-48ee-9e0a-2e03f2af8d44 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:22.790768 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 407/800] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:22.792593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba5aec-7ef2-48ee-9e0a-2e03f2af8d44 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:22.830349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b281554-ba01-4842-bf8b-ea4716adf71c None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:22.874315 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ea617-9617-4631-a8e0-59705bc155b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:22.875295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ea617-9617-4631-a8e0-59705bc155b7 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:22.875731 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ea617-9617-4631-a8e0-59705bc155b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:22.876157 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ea617-9617-4631-a8e0-59705bc155b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:22.886930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ea617-9617-4631-a8e0-59705bc155b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:22.890569 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ea617-9617-4631-a8e0-59705bc155b7 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:22.915062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-861ea617-9617-4631-a8e0-59705bc155b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:22.915062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:22.915062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:22.915062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:22.915062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:22.915062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:22.915062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:22.915062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:22.915062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:22.915062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:22.915062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:22.915062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:22.915062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:22.915062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:22.915062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:22.916151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:22.917241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:22.917241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:22.917241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:22.917241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:22.917241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:22.917241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:22.917241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:22.917241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:22.920247 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 394/801] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:22.933858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9cf9f-6e8c-40e2-b565-03476f8b7d25 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:22.934535 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9cf9f-6e8c-40e2-b565-03476f8b7d25 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:22.935142 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9cf9f-6e8c-40e2-b565-03476f8b7d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:22.970423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-4bd9cf9f-6e8c-40e2-b565-03476f8b7d25 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:22.973305 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4bd9cf9f-6e8c-40e2-b565-03476f8b7d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:22.983297 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba5aec-7ef2-48ee-9e0a-2e03f2af8d44 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:23.026858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad6174-f0d5-4875-8087-75fe6e992703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:23.027636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad6174-f0d5-4875-8087-75fe6e992703 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:23.027925 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad6174-f0d5-4875-8087-75fe6e992703 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:23.028204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad6174-f0d5-4875-8087-75fe6e992703 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:23.028925 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad6174-f0d5-4875-8087-75fe6e992703 None admin] RBAC: Adding query filter params (name=network) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:23.029544 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad6174-f0d5-4875-8087-75fe6e992703 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:23.031757 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad6174-f0d5-4875-8087-75fe6e992703 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:23.038098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 408/802] 10.222.0.88 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:23.038986 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad6174-f0d5-4875-8087-75fe6e992703 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:23.079408 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:23.117186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 395/803] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:23.118865 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9cf9f-6e8c-40e2-b565-03476f8b7d25 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:23.162997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:23.212181 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad6174-f0d5-4875-8087-75fe6e992703 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:23.281840 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edee032-c22d-4298-9a9e-b3286ca6bb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:23.283278 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edee032-c22d-4298-9a9e-b3286ca6bb14 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:23.283979 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edee032-c22d-4298-9a9e-b3286ca6bb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:23.284664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edee032-c22d-4298-9a9e-b3286ca6bb14 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:23.286221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edee032-c22d-4298-9a9e-b3286ca6bb14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:23.287166 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edee032-c22d-4298-9a9e-b3286ca6bb14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:23.289985 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edee032-c22d-4298-9a9e-b3286ca6bb14 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:23.316330 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 409/804] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:23.326773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edee032-c22d-4298-9a9e-b3286ca6bb14 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:23.336386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9cf9f-6e8c-40e2-b565-03476f8b7d25 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:23.406554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238590e-1a5f-43d1-af5c-846e17f34eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:23.407705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238590e-1a5f-43d1-af5c-846e17f34eaf None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:23.408278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238590e-1a5f-43d1-af5c-846e17f34eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:23.408838 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238590e-1a5f-43d1-af5c-846e17f34eaf None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:23.413096 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238590e-1a5f-43d1-af5c-846e17f34eaf None admin] RBAC: Adding query filter params (type=network) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:23.413096 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238590e-1a5f-43d1-af5c-846e17f34eaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:23.413096 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238590e-1a5f-43d1-af5c-846e17f34eaf None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:23.429086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 396/805] 10.222.0.88 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:23.432524 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238590e-1a5f-43d1-af5c-846e17f34eaf None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:23.556521 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:23.579330 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edee032-c22d-4298-9a9e-b3286ca6bb14 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:23.665031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980aeed-2a4c-4047-ab3f-2a4c57651ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:23.666301 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980aeed-2a4c-4047-ab3f-2a4c57651ba2 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:23.667229 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980aeed-2a4c-4047-ab3f-2a4c57651ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:23.667747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980aeed-2a4c-4047-ab3f-2a4c57651ba2 None admin] PATH_INFO: `/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/601a3defca70428689ee22517d7bcb19/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:23.671910 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980aeed-2a4c-4047-ab3f-2a4c57651ba2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a7317af325f485e8fae49631cb5bc20, user_id=601a3defca70428689ee22517d7bcb19, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:23.676902 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980aeed-2a4c-4047-ab3f-2a4c57651ba2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:23.707083 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:23.711039 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 410/806] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:10:23 2022] PUT /identity/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/601a3defca70428689ee22517d7bcb19/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:10:23.712576 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980aeed-2a4c-4047-ab3f-2a4c57651ba2 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:23.728450 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238590e-1a5f-43d1-af5c-846e17f34eaf None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:23.829753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59b255-e328-4e36-aaa9-0aa517356b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:23.832204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59b255-e328-4e36-aaa9-0aa517356b1a None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:23.833814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59b255-e328-4e36-aaa9-0aa517356b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:23.834507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59b255-e328-4e36-aaa9-0aa517356b1a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:23.836109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59b255-e328-4e36-aaa9-0aa517356b1a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:23.839194 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59b255-e328-4e36-aaa9-0aa517356b1a None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:23.892081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 397/807] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:10:23 2022] POST /identity/v3/endpoints => generated 337 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:24.000893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980aeed-2a4c-4047-ab3f-2a4c57651ba2 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:24.094001 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582511f7-8403-4b5d-9b41-879c7d1dcfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:24.095168 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582511f7-8403-4b5d-9b41-879c7d1dcfd3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:24.095639 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582511f7-8403-4b5d-9b41-879c7d1dcfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:24.096091 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582511f7-8403-4b5d-9b41-879c7d1dcfd3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:24.117095 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582511f7-8403-4b5d-9b41-879c7d1dcfd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:24.127468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-582511f7-8403-4b5d-9b41-879c7d1dcfd3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:24.127468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:24.127468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:24.127468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:24.127468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:24.127468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:24.127468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:24.127468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:24.127468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:24.127468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:24.127468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:24.127468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:24.127468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:24.127468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:24.127468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:24.127468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:24.129162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:24.129162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:24.129162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:24.129162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:24.129162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:24.129162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:24.129162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:24.129162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:24.129162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:24.129162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:24.129162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:24.129162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:24.129162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:24.129162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:24.129162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:24.129162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:24.130754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:24.130754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:24.130754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:24.130754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:24.130754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:24.130754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:24.130754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:24.130754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:24.130754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:24.130754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:24.130754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:24.130754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:24.130754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 411/808] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:24.140848 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59b255-e328-4e36-aaa9-0aa517356b1a None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:24.370710 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59b255-e328-4e36-aaa9-0aa517356b1a None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:24.438657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129195c-50e0-45c9-90e4-8c2398d1a21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:24.440212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129195c-50e0-45c9-90e4-8c2398d1a21a None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:24.440940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129195c-50e0-45c9-90e4-8c2398d1a21a None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:24.441729 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129195c-50e0-45c9-90e4-8c2398d1a21a None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:24.443178 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129195c-50e0-45c9-90e4-8c2398d1a21a None admin] RBAC: Adding query filter params (name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:24.443986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129195c-50e0-45c9-90e4-8c2398d1a21a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:24.446848 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129195c-50e0-45c9-90e4-8c2398d1a21a None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:24.464216 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 398/809] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:24.472843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582511f7-8403-4b5d-9b41-879c7d1dcfd3 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:24.491322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:24.688195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582511f7-8403-4b5d-9b41-879c7d1dcfd3 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:24.708050 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cb41e-67c5-4322-af22-bc2cf4329656 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:24.708398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cb41e-67c5-4322-af22-bc2cf4329656 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:24.708595 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cb41e-67c5-4322-af22-bc2cf4329656 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:24.710372 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 399/810] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:24.716855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656836a5-87d8-4e3d-8d0b-cbc68a81e063 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:24.717214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656836a5-87d8-4e3d-8d0b-cbc68a81e063 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:24.717536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656836a5-87d8-4e3d-8d0b-cbc68a81e063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:24.737755 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:24.746417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-656836a5-87d8-4e3d-8d0b-cbc68a81e063 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:24.748084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-656836a5-87d8-4e3d-8d0b-cbc68a81e063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:24.760479 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98daa6b-c8fe-4b0c-af68-029f89945060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:24.761999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98daa6b-c8fe-4b0c-af68-029f89945060 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:24.762294 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98daa6b-c8fe-4b0c-af68-029f89945060 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:24.762548 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98daa6b-c8fe-4b0c-af68-029f89945060 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:24.790970 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98daa6b-c8fe-4b0c-af68-029f89945060 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:24.792555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98daa6b-c8fe-4b0c-af68-029f89945060 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:24.826186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-a98daa6b-c8fe-4b0c-af68-029f89945060 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:24.826186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:24.826186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:24.826186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:24.826186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:24.826186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:24.826186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:24.826186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:24.826186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:24.826186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:24.826186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:24.826186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:24.826186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:24.826186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:24.826186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:24.826186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:24.827354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:24.828749 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:24.828749 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:24.828749 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:24.829461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 412/811] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:24.831638 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a35b0-5263-4d17-ab4e-230238c2e355 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:24.832146 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a35b0-5263-4d17-ab4e-230238c2e355 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:24.832630 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a35b0-5263-4d17-ab4e-230238c2e355 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:24.835320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 413/812] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:24.836961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a35b0-5263-4d17-ab4e-230238c2e355 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:24.882164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 400/813] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:24.883911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc59bf-5fe3-44c8-a583-f6eae7679b50 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:24.884223 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc59bf-5fe3-44c8-a583-f6eae7679b50 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:24.884483 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc59bf-5fe3-44c8-a583-f6eae7679b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:24.905091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-e2cc59bf-5fe3-44c8-a583-f6eae7679b50 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:24.907547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e2cc59bf-5fe3-44c8-a583-f6eae7679b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:24.982511 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:25.017657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 401/814] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:25.075601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a35b0-5263-4d17-ab4e-230238c2e355 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:25.140602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d240a0e-81db-466b-9bd4-a3519cf40e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:25.143306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d240a0e-81db-466b-9bd4-a3519cf40e8b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:25.144041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d240a0e-81db-466b-9bd4-a3519cf40e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:25.144777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d240a0e-81db-466b-9bd4-a3519cf40e8b None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:25.146195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d240a0e-81db-466b-9bd4-a3519cf40e8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:25.146981 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d240a0e-81db-466b-9bd4-a3519cf40e8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:25.149716 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d240a0e-81db-466b-9bd4-a3519cf40e8b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:25.165872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc59bf-5fe3-44c8-a583-f6eae7679b50 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:25.176957 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 414/815] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:25.184855 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d240a0e-81db-466b-9bd4-a3519cf40e8b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:25.293292 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:25.339561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d240a0e-81db-466b-9bd4-a3519cf40e8b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:25.346968 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc59bf-5fe3-44c8-a583-f6eae7679b50 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:25.372819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:25.393137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ca8ca-d950-4f0a-b4ef-a2c0bfce2d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:25.393814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ca8ca-d950-4f0a-b4ef-a2c0bfce2d67 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:25.394156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ca8ca-d950-4f0a-b4ef-a2c0bfce2d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:25.394365 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ca8ca-d950-4f0a-b4ef-a2c0bfce2d67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:25.407482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ca8ca-d950-4f0a-b4ef-a2c0bfce2d67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:25.410497 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20014aab-acbb-4a15-9410-de5151e04b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:25.411616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20014aab-acbb-4a15-9410-de5151e04b2c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:25.412094 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20014aab-acbb-4a15-9410-de5151e04b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:25.412538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20014aab-acbb-4a15-9410-de5151e04b2c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:25.412669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-767ca8ca-d950-4f0a-b4ef-a2c0bfce2d67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:25.412669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:25.412669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:25.412669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:25.412669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:25.412669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:25.412669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:25.412669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:25.412669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:25.412669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:25.412669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:25.412669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:25.412669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:25.412669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:25.412669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:25.412669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:25.413799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:25.413799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:25.413799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:25.413799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:25.413799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:25.413799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:25.413799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:25.413799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:25.413799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:25.413799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:25.413799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:25.413799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:25.413799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:25.413799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:25.413799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:25.413799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:25.414640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:25.414640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:25.414640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:25.414640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:25.414640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:25.414640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:25.414640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:25.414640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:25.414640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:25.414640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:25.414640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:25.414640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:25.414640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 402/816] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:25.414640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ca8ca-d950-4f0a-b4ef-a2c0bfce2d67 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:25.425542 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20014aab-acbb-4a15-9410-de5151e04b2c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:25.427018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20014aab-acbb-4a15-9410-de5151e04b2c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:25.441621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-20014aab-acbb-4a15-9410-de5151e04b2c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:10:25.441621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:25.441621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:25.441621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:25.441621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:25.441621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:25.441621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:25.441621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:25.441621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:25.441621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:25.441621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:25.441621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:25.441621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:25.441621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:25.441621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:25.441621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:25.443480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:25.445299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:25.445299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:25.445299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:25.445299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:25.445299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:25.445299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:25.445299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:25.445299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:25.445299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:10:25.445299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:25.445299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 415/817] 10.222.0.88 () {56 vars in 1117 bytes} [Tue Aug 23 22:10:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:25.445299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20014aab-acbb-4a15-9410-de5151e04b2c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:25.584289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ca8ca-d950-4f0a-b4ef-a2c0bfce2d67 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:25.605902 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20014aab-acbb-4a15-9410-de5151e04b2c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:25.614656 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:25.626460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d2785-e03e-4925-8cea-5b03075d6946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:25.627213 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d2785-e03e-4925-8cea-5b03075d6946 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:25.627571 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d2785-e03e-4925-8cea-5b03075d6946 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:25.627919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d2785-e03e-4925-8cea-5b03075d6946 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:25.628595 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d2785-e03e-4925-8cea-5b03075d6946 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:25.630112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d2785-e03e-4925-8cea-5b03075d6946 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-639d2785-e03e-4925-8cea-5b03075d6946 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:25.640745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:25.642084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:25.642084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:25.642084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:25.642084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:25.642084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:25.642084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:25.642084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:25.642084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:25.642084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:25.642084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:25.642084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:25.642084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:25.642084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:25.642084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:25.642084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:25.642084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:25.643127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:25.643127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:25.643127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:25.643127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:25.643127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:25.643127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:25.643127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:25.643127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:25.643127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:25.643127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:25.643127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:25.643127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:25.643127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:10:25.643127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:25.649201 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 403/818] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:25.649201 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d2785-e03e-4925-8cea-5b03075d6946 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:25.666466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5e800-1fa1-4a90-8043-2075394d421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:25.667262 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5e800-1fa1-4a90-8043-2075394d421e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:25.667648 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5e800-1fa1-4a90-8043-2075394d421e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:25.668030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5e800-1fa1-4a90-8043-2075394d421e None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:25.668810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5e800-1fa1-4a90-8043-2075394d421e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:25.669298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5e800-1fa1-4a90-8043-2075394d421e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:25.670733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5e800-1fa1-4a90-8043-2075394d421e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:25.685611 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 416/819] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:25.685611 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5e800-1fa1-4a90-8043-2075394d421e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:25.757418 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:25.795658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d2785-e03e-4925-8cea-5b03075d6946 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:25.846820 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5e800-1fa1-4a90-8043-2075394d421e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:25.861547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175d249-8e38-4248-8b60-104174d88868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:25.862490 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175d249-8e38-4248-8b60-104174d88868 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:25.862919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175d249-8e38-4248-8b60-104174d88868 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:25.863340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175d249-8e38-4248-8b60-104174d88868 None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:25.864592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175d249-8e38-4248-8b60-104174d88868 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:25.865132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175d249-8e38-4248-8b60-104174d88868 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:25.867703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175d249-8e38-4248-8b60-104174d88868 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:25.886294 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea603e-6578-49db-bf69-03c57537f9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:25.886942 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea603e-6578-49db-bf69-03c57537f9a8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:25.887221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea603e-6578-49db-bf69-03c57537f9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:25.887495 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea603e-6578-49db-bf69-03c57537f9a8 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:25.888126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea603e-6578-49db-bf69-03c57537f9a8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:25.888426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea603e-6578-49db-bf69-03c57537f9a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:25.890403 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea603e-6578-49db-bf69-03c57537f9a8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:25.900487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 417/820] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:25.900487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea603e-6578-49db-bf69-03c57537f9a8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:25.903506 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 404/821] 10.222.0.88 () {56 vars in 1149 bytes} [Tue Aug 23 22:10:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:25.904493 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175d249-8e38-4248-8b60-104174d88868 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:25.964307 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:26.031177 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175d249-8e38-4248-8b60-104174d88868 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:26.075731 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea603e-6578-49db-bf69-03c57537f9a8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:26.076793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e65d3c-8e2a-40fc-b181-7254c7709c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:26.077976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e65d3c-8e2a-40fc-b181-7254c7709c4a None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:26.078713 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e65d3c-8e2a-40fc-b181-7254c7709c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:26.078713 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e65d3c-8e2a-40fc-b181-7254c7709c4a None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:26.079221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e65d3c-8e2a-40fc-b181-7254c7709c4a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:26.079519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e65d3c-8e2a-40fc-b181-7254c7709c4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:26.080925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e65d3c-8e2a-40fc-b181-7254c7709c4a None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:26.081832 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:26.086559 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 405/822] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:25 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:26.087419 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e65d3c-8e2a-40fc-b181-7254c7709c4a None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:26.161217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdae3e-36a0-478a-a88c-3cc9d03a3b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:26.163335 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdae3e-36a0-478a-a88c-3cc9d03a3b5e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:26.165184 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdae3e-36a0-478a-a88c-3cc9d03a3b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:26.166150 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdae3e-36a0-478a-a88c-3cc9d03a3b5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:26.190858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdae3e-36a0-478a-a88c-3cc9d03a3b5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:26.192771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdae3e-36a0-478a-a88c-3cc9d03a3b5e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:26.217019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-b6bdae3e-36a0-478a-a88c-3cc9d03a3b5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:26.217019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:26.217019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:26.217019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:26.217019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:26.217019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:26.217019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:26.217019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:26.217019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:26.217019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:26.217019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:26.217019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:26.217019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:26.217019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:26.217019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:26.217019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:26.219427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:26.221092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:26.221092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:26.221092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:26.224554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 418/823] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:26.225570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdae3e-36a0-478a-a88c-3cc9d03a3b5e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:26.256887 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e65d3c-8e2a-40fc-b181-7254c7709c4a None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:26.273304 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:26.304488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c48def-b96f-4c81-be3a-608699ff1e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:26.305577 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48def-b96f-4c81-be3a-608699ff1e94 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:26.306059 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48def-b96f-4c81-be3a-608699ff1e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:26.306582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48def-b96f-4c81-be3a-608699ff1e94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:26.320152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c48def-b96f-4c81-be3a-608699ff1e94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:26.321733 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c48def-b96f-4c81-be3a-608699ff1e94 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:26.340412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-f6c48def-b96f-4c81-be3a-608699ff1e94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:26.340412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:26.340412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:26.340412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:26.340412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:26.340412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:26.340412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:26.340412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:26.340412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:26.340412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:26.340412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:26.340412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:26.340412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:26.340412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:26.340412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:26.340412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:26.341588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:26.342583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:26.342583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:26.342583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:26.342583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 406/824] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:26.351635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c48def-b96f-4c81-be3a-608699ff1e94 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:26.430536 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdae3e-36a0-478a-a88c-3cc9d03a3b5e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:26.498283 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cfeeb-0fd2-407a-84c0-f5a072c8e970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:26.500351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cfeeb-0fd2-407a-84c0-f5a072c8e970 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:26.501204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cfeeb-0fd2-407a-84c0-f5a072c8e970 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:26.501996 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cfeeb-0fd2-407a-84c0-f5a072c8e970 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:26.503405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92cfeeb-0fd2-407a-84c0-f5a072c8e970 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:26.504192 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92cfeeb-0fd2-407a-84c0-f5a072c8e970 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:26.507166 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92cfeeb-0fd2-407a-84c0-f5a072c8e970 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:26.526082 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c48def-b96f-4c81-be3a-608699ff1e94 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:26.533269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 419/825] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:26.544614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cfeeb-0fd2-407a-84c0-f5a072c8e970 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:26.565487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:26.601811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870b075-5f5f-44ca-9642-7170107fd244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:26.602869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870b075-5f5f-44ca-9642-7170107fd244 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:26.603376 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870b075-5f5f-44ca-9642-7170107fd244 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:26.603878 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870b075-5f5f-44ca-9642-7170107fd244 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:26.605132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870b075-5f5f-44ca-9642-7170107fd244 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:26.605704 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870b075-5f5f-44ca-9642-7170107fd244 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:26.608203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870b075-5f5f-44ca-9642-7170107fd244 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:26.633898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 407/826] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:26.645943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870b075-5f5f-44ca-9642-7170107fd244 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:26.709529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:26.739210 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cfeeb-0fd2-407a-84c0-f5a072c8e970 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:26.813853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e513e6b-2993-43f5-8f94-b8a08e74860d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:26.815547 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e513e6b-2993-43f5-8f94-b8a08e74860d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:26.816367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e513e6b-2993-43f5-8f94-b8a08e74860d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:26.817046 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e513e6b-2993-43f5-8f94-b8a08e74860d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:26.819573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870b075-5f5f-44ca-9642-7170107fd244 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:26.831660 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e513e6b-2993-43f5-8f94-b8a08e74860d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:26.834642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e513e6b-2993-43f5-8f94-b8a08e74860d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:26.859719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-0e513e6b-2993-43f5-8f94-b8a08e74860d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:26.859719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:26.859719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:26.859719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:26.859719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:26.859719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:26.859719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:26.859719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:26.859719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:26.859719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:26.859719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:26.859719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:26.859719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:26.859719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:26.859719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:26.859719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:26.861809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:26.863730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:26.863730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:26.863730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:26.863730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:26.863730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:26.863730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:26.863730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:26.863730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:26.863730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:26.863730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:26.867262 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 420/827] 10.222.0.88 () {56 vars in 1093 bytes} [Tue Aug 23 22:10:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:26.879973 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e513e6b-2993-43f5-8f94-b8a08e74860d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:26.894962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e4c2c-38b8-4cd0-a4d8-4246a6d02d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:26.895631 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e4c2c-38b8-4cd0-a4d8-4246a6d02d00 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:26.895912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e4c2c-38b8-4cd0-a4d8-4246a6d02d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:26.896174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e4c2c-38b8-4cd0-a4d8-4246a6d02d00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:26.904112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e4c2c-38b8-4cd0-a4d8-4246a6d02d00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:26.906351 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e4c2c-38b8-4cd0-a4d8-4246a6d02d00 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:26.919889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-4c2e4c2c-38b8-4cd0-a4d8-4246a6d02d00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:26.919889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:26.919889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:26.919889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:26.919889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:26.919889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:26.919889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:26.919889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:26.919889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:26.919889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:26.919889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:26.919889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:26.919889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:26.919889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:26.919889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:26.922364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:26.924346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:26.924346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:26.924346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:26.924346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:26.924346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:26.924346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:26.924346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:26.924346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:26.924346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 408/828] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:26.924346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bff4e-aba7-4c6b-bfde-81e132de700d None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:26.924346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bff4e-aba7-4c6b-bfde-81e132de700d None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:26.927199 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bff4e-aba7-4c6b-bfde-81e132de700d None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:26.929510 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 409/829] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:26.930834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958f210-08cf-4a7e-ab44-e22593b28f82 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:26.931101 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958f210-08cf-4a7e-ab44-e22593b28f82 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:26.931374 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958f210-08cf-4a7e-ab44-e22593b28f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:26.954076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-7958f210-08cf-4a7e-ab44-e22593b28f82 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:26.954923 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-7958f210-08cf-4a7e-ab44-e22593b28f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:27.039501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 410/830] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:27.044244 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958f210-08cf-4a7e-ab44-e22593b28f82 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:27.050274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:27.086349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:27.103608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e513e6b-2993-43f5-8f94-b8a08e74860d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:27.167682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f8f88-4e22-4a9f-8625-b8781785a110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:27.169495 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f8f88-4e22-4a9f-8625-b8781785a110 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:27.170034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f8f88-4e22-4a9f-8625-b8781785a110 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:27.170517 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f8f88-4e22-4a9f-8625-b8781785a110 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:27.171582 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f8f88-4e22-4a9f-8625-b8781785a110 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:27.172110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f8f88-4e22-4a9f-8625-b8781785a110 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:27.174190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f8f88-4e22-4a9f-8625-b8781785a110 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:27.196825 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 421/831] 10.222.0.88 () {56 vars in 1133 bytes} [Tue Aug 23 22:10:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:27.206008 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f8f88-4e22-4a9f-8625-b8781785a110 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:27.290863 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958f210-08cf-4a7e-ab44-e22593b28f82 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:27.338564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f8f88-4e22-4a9f-8625-b8781785a110 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:27.355510 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde799fa-f4da-4718-a328-83d5ba98e7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:27.356581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde799fa-f4da-4718-a328-83d5ba98e7fb None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:27.357037 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde799fa-f4da-4718-a328-83d5ba98e7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:27.357517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde799fa-f4da-4718-a328-83d5ba98e7fb None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:27.359618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde799fa-f4da-4718-a328-83d5ba98e7fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:27.360140 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde799fa-f4da-4718-a328-83d5ba98e7fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:27.362700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde799fa-f4da-4718-a328-83d5ba98e7fb None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:27.387113 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01809c-445d-40d1-8085-35e9e31301a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:27.387387 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 411/832] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:27.388686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01809c-445d-40d1-8085-35e9e31301a2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:27.389377 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01809c-445d-40d1-8085-35e9e31301a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:27.390682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01809c-445d-40d1-8085-35e9e31301a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:27.396161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde799fa-f4da-4718-a328-83d5ba98e7fb None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:27.406208 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:27.511207 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01809c-445d-40d1-8085-35e9e31301a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:27.512800 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01809c-445d-40d1-8085-35e9e31301a2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:27.548160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:27.588944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde799fa-f4da-4718-a328-83d5ba98e7fb None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:27.641181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69dbf3-46f5-4d49-a19f-f28f1cd430dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:27.641840 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69dbf3-46f5-4d49-a19f-f28f1cd430dd None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:27.642125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69dbf3-46f5-4d49-a19f-f28f1cd430dd None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:27.642391 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69dbf3-46f5-4d49-a19f-f28f1cd430dd None admin] PATH_INFO: `/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/cb001b14eec54047aee42ca6bd4a1eba/roles/23ea214daa734346a6da3a0a8c50d23a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:27.667024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69dbf3-46f5-4d49-a19f-f28f1cd430dd None admin] RBAC: Authorizing `identity:create_grant(project_id=8a7317af325f485e8fae49631cb5bc20, user_id=cb001b14eec54047aee42ca6bd4a1eba, role_id=23ea214daa734346a6da3a0a8c50d23a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:27.668830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69dbf3-46f5-4d49-a19f-f28f1cd430dd None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:27.689646 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 422/833] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:27 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 22:10:27.690999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01809c-445d-40d1-8085-35e9e31301a2 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:27.698566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 412/834] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:10:27 2022] PUT /identity/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/cb001b14eec54047aee42ca6bd4a1eba/roles/23ea214daa734346a6da3a0a8c50d23a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:10:27.700135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69dbf3-46f5-4d49-a19f-f28f1cd430dd None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:27.747258 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:27.856844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69dbf3-46f5-4d49-a19f-f28f1cd430dd None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:27.894821 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01809c-445d-40d1-8085-35e9e31301a2 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:27.902188 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:27.922517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428233b-e517-4699-84b4-d59a665e3a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:27.923772 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428233b-e517-4699-84b4-d59a665e3a4d None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:27.924224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428233b-e517-4699-84b4-d59a665e3a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:27.924663 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428233b-e517-4699-84b4-d59a665e3a4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:27.945495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428233b-e517-4699-84b4-d59a665e3a4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:27.948122 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428233b-e517-4699-84b4-d59a665e3a4d None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:27.973799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-c428233b-e517-4699-84b4-d59a665e3a4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:27.973799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:27.973799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:27.973799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:27.973799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:27.973799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:27.973799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:27.973799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:27.973799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:27.973799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:27.973799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:27.973799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:27.973799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:27.973799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:27.973799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:27.973799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:27.975462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:27.978608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:27.978608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:27.978608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:27.978608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 413/835] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:27.988655 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93305ca-f29f-4680-9eef-1664dd418ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:27.989475 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428233b-e517-4699-84b4-d59a665e3a4d None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:27.989740 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93305ca-f29f-4680-9eef-1664dd418ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:27.990298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93305ca-f29f-4680-9eef-1664dd418ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:27.990755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93305ca-f29f-4680-9eef-1664dd418ed4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.017739 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93305ca-f29f-4680-9eef-1664dd418ed4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:28.028411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-c93305ca-f29f-4680-9eef-1664dd418ed4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:28.028411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:28.028411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:28.028411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:28.028411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:28.028411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:28.028411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:28.028411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:28.028411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:28.028411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:28.028411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:28.028411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:28.028411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:28.028411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:28.028411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:28.028411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:28.032427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:28.032427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:28.032427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:28.032427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:28.032427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:28.032427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:28.032427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:28.032427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:28.032427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:28.032427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:28.032427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:28.032427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:28.032427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:28.032427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:28.032427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:28.032427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:28.034808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:28.034808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:28.034808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:28.034808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:28.034808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:28.034808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:28.034808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:28.034808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:28.034808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:28.034808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:28.034808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:28.034808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:28.037261 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 423/836] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:28.042268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93305ca-f29f-4680-9eef-1664dd418ed4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:28.181148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428233b-e517-4699-84b4-d59a665e3a4d None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:28.224583 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:28.236258 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac9079-bdbb-49e4-b8a2-b03d43f7a23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:28.237452 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93305ca-f29f-4680-9eef-1664dd418ed4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:28.237898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac9079-bdbb-49e4-b8a2-b03d43f7a23e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.238665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac9079-bdbb-49e4-b8a2-b03d43f7a23e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.239322 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac9079-bdbb-49e4-b8a2-b03d43f7a23e None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.241202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac9079-bdbb-49e4-b8a2-b03d43f7a23e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:28.243422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac9079-bdbb-49e4-b8a2-b03d43f7a23e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:28.246034 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac9079-bdbb-49e4-b8a2-b03d43f7a23e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:28.274066 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 414/837] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:28.275113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac9079-bdbb-49e4-b8a2-b03d43f7a23e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:28.294808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc848b09-94c2-40e4-b9c7-eec845717be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:28.297892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc848b09-94c2-40e4-b9c7-eec845717be3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.298644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc848b09-94c2-40e4-b9c7-eec845717be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.299934 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc848b09-94c2-40e4-b9c7-eec845717be3 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.301441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc848b09-94c2-40e4-b9c7-eec845717be3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:28.302181 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc848b09-94c2-40e4-b9c7-eec845717be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:28.307785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc848b09-94c2-40e4-b9c7-eec845717be3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:28.331915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:28.333744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 424/838] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:28.334590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc848b09-94c2-40e4-b9c7-eec845717be3 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:28.433296 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac9079-bdbb-49e4-b8a2-b03d43f7a23e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:28.487267 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172544c-71ad-4ea3-bdb7-3e02ee90de04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:28.487924 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172544c-71ad-4ea3-bdb7-3e02ee90de04 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.488213 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172544c-71ad-4ea3-bdb7-3e02ee90de04 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.488519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172544c-71ad-4ea3-bdb7-3e02ee90de04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.496148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172544c-71ad-4ea3-bdb7-3e02ee90de04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:28.497706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172544c-71ad-4ea3-bdb7-3e02ee90de04 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:28.508507 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc848b09-94c2-40e4-b9c7-eec845717be3 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:28.511038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-d172544c-71ad-4ea3-bdb7-3e02ee90de04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:28.511038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:28.511038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:28.511038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:28.511038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:28.511038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:28.511038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:28.511038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:28.511038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:28.511038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:28.511038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:28.511038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:28.511038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:28.511038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:28.511038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:28.512547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:28.514581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:28.514581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:28.514581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:28.514581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:28.514581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:28.514581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:28.514581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:28.514581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:28.514581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 415/839] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:28.514581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27d463-b555-4ab4-bab2-e501e746ab0b None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.514581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27d463-b555-4ab4-bab2-e501e746ab0b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.516732 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27d463-b555-4ab4-bab2-e501e746ab0b None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.516732 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 416/840] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:28.521264 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27d463-b555-4ab4-bab2-e501e746ab0b None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:28.573644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771ca05-53fa-4101-ae1d-f38e7bcd4c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:28.574402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771ca05-53fa-4101-ae1d-f38e7bcd4c66 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.574682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771ca05-53fa-4101-ae1d-f38e7bcd4c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.574955 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771ca05-53fa-4101-ae1d-f38e7bcd4c66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.587922 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771ca05-53fa-4101-ae1d-f38e7bcd4c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:28.589902 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771ca05-53fa-4101-ae1d-f38e7bcd4c66 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:28.605732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-3771ca05-53fa-4101-ae1d-f38e7bcd4c66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:28.605732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:28.605732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:28.605732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:28.605732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:28.605732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:28.605732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:28.605732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:28.605732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:28.605732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:28.605732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:28.605732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:28.605732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:28.605732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:28.605732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:28.605732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:28.607359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:28.608555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:28.608555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:28.608555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:28.608555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 425/841] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:28.608555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d430008-2654-4624-b76a-62fc774fdaf9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.608555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d430008-2654-4624-b76a-62fc774fdaf9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.608977 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d430008-2654-4624-b76a-62fc774fdaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.629082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9d430008-2654-4624-b76a-62fc774fdaf9 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:28.630322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9d430008-2654-4624-b76a-62fc774fdaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:28.632662 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:28.668271 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:28.680046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27d463-b555-4ab4-bab2-e501e746ab0b None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:28.718353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 426/842] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:28.719213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d430008-2654-4624-b76a-62fc774fdaf9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:28.735381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cd28f-5040-4096-be20-7fb40ff9b257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:28.736757 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cd28f-5040-4096-be20-7fb40ff9b257 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.737476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cd28f-5040-4096-be20-7fb40ff9b257 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.738166 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cd28f-5040-4096-be20-7fb40ff9b257 None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.739619 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cd28f-5040-4096-be20-7fb40ff9b257 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:28.740361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cd28f-5040-4096-be20-7fb40ff9b257 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:28.743031 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cd28f-5040-4096-be20-7fb40ff9b257 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:28.774590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 417/843] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:28.781756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cd28f-5040-4096-be20-7fb40ff9b257 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:28.919086 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d430008-2654-4624-b76a-62fc774fdaf9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:28.959212 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e6b9e-7990-45d9-815c-e9e88a785182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:28.959936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e6b9e-7990-45d9-815c-e9e88a785182 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.960272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e6b9e-7990-45d9-815c-e9e88a785182 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.960604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e6b9e-7990-45d9-815c-e9e88a785182 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.961287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e6b9e-7990-45d9-815c-e9e88a785182 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:28.961602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e6b9e-7990-45d9-815c-e9e88a785182 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:28.963052 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e6b9e-7990-45d9-815c-e9e88a785182 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:28.972809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:28.980361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 427/844] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:28.988401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e6b9e-7990-45d9-815c-e9e88a785182 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:28.991064 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cd28f-5040-4096-be20-7fb40ff9b257 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:29.055127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55293872-43e3-4a2a-b5a4-5e1380754e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:29.056159 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55293872-43e3-4a2a-b5a4-5e1380754e26 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.056658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55293872-43e3-4a2a-b5a4-5e1380754e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.057191 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55293872-43e3-4a2a-b5a4-5e1380754e26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.058539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55293872-43e3-4a2a-b5a4-5e1380754e26 None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=e3517b3b077a4a1fa76163657b46930c) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:29.059072 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55293872-43e3-4a2a-b5a4-5e1380754e26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:29.061870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55293872-43e3-4a2a-b5a4-5e1380754e26 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:29.081688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 418/845] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:28 2022] GET /identity/v3/role_assignments?user.id=e3517b3b077a4a1fa76163657b46930c&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=dc5d71ab685e420697f520aa3a368177 => generated 608 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:29.083091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55293872-43e3-4a2a-b5a4-5e1380754e26 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:29.110764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e6b9e-7990-45d9-815c-e9e88a785182 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:29.113243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:29.159066 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a2493-df67-46a2-a5a2-4630178b1646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:29.159721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a2493-df67-46a2-a5a2-4630178b1646 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.160039 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a2493-df67-46a2-a5a2-4630178b1646 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.160277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a2493-df67-46a2-a5a2-4630178b1646 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.170689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a2493-df67-46a2-a5a2-4630178b1646 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:29.173198 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a2493-df67-46a2-a5a2-4630178b1646 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:29.192365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-fd1a2493-df67-46a2-a5a2-4630178b1646 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:29.192365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:29.192365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:29.192365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:29.192365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:29.192365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:29.192365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:29.192365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:29.192365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:29.192365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:29.192365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:29.192365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:29.192365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:29.192365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:29.192365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:29.192365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:29.194506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:29.196926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:29.196926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:29.196926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:29.196926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:29.196926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:29.196926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:29.196926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:29.196926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:29.196926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:29.196926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:29.196926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 428/846] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:29.201535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a2493-df67-46a2-a5a2-4630178b1646 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:29.286086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55293872-43e3-4a2a-b5a4-5e1380754e26 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:29.329495 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:29.351013 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55157dcf-b818-48e4-84d5-8e01b6c52344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:29.352462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55157dcf-b818-48e4-84d5-8e01b6c52344 None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.352462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55157dcf-b818-48e4-84d5-8e01b6c52344 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.352904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55157dcf-b818-48e4-84d5-8e01b6c52344 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.353760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55157dcf-b818-48e4-84d5-8e01b6c52344 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:29.355748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55157dcf-b818-48e4-84d5-8e01b6c52344 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:29.385039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 419/847] 10.222.0.88 () {60 vars in 1141 bytes} [Tue Aug 23 22:10:29 2022] POST /identity/v3/services => generated 251 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:29.388502 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a2493-df67-46a2-a5a2-4630178b1646 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:29.474239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce28f9-77cb-4f56-bd30-ef820cdd3853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:29.474884 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce28f9-77cb-4f56-bd30-ef820cdd3853 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.475167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce28f9-77cb-4f56-bd30-ef820cdd3853 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.475439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce28f9-77cb-4f56-bd30-ef820cdd3853 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.476199 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce28f9-77cb-4f56-bd30-ef820cdd3853 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:29.476631 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce28f9-77cb-4f56-bd30-ef820cdd3853 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:29.478249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce28f9-77cb-4f56-bd30-ef820cdd3853 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:29.508162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 429/848] 10.222.0.88 () {56 vars in 1135 bytes} [Tue Aug 23 22:10:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:29.513865 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55157dcf-b818-48e4-84d5-8e01b6c52344 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:29.624166 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:29.653106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55157dcf-b818-48e4-84d5-8e01b6c52344 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:29.710514 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5ec11-6ba5-4ceb-a5c5-f485bea5cc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:29.711931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5ec11-6ba5-4ceb-a5c5-f485bea5cc7b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.712632 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5ec11-6ba5-4ceb-a5c5-f485bea5cc7b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.713485 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5ec11-6ba5-4ceb-a5c5-f485bea5cc7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.854959 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5ec11-6ba5-4ceb-a5c5-f485bea5cc7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:29.858066 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5ec11-6ba5-4ceb-a5c5-f485bea5cc7b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:30.015707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:30.060146 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 420/849] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:29 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:10:30.069716 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce28f9-77cb-4f56-bd30-ef820cdd3853 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:30.186131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce28f9-77cb-4f56-bd30-ef820cdd3853 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:30.205613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:30.228582 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa19276-2873-4fe3-82a7-2a31ded4e249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:30.229331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa19276-2873-4fe3-82a7-2a31ded4e249 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.229614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa19276-2873-4fe3-82a7-2a31ded4e249 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.229885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa19276-2873-4fe3-82a7-2a31ded4e249 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.242114 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa19276-2873-4fe3-82a7-2a31ded4e249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:30.242798 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c36f03-3e8b-42f8-b3bb-274a21e5e8fb None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.243296 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c36f03-3e8b-42f8-b3bb-274a21e5e8fb None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.243590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa19276-2873-4fe3-82a7-2a31ded4e249 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:30.243750 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c36f03-3e8b-42f8-b3bb-274a21e5e8fb None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.252324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 421/850] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:30 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:30.257019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5c26d-4378-42cb-9b3d-8c261c4c975d None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.257530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5c26d-4378-42cb-9b3d-8c261c4c975d None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.258012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5c26d-4378-42cb-9b3d-8c261c4c975d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-3fa19276-2873-4fe3-82a7-2a31ded4e249 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:30.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:30.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:30.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:30.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:30.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:30.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:30.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:30.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:30.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:30.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:30.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:30.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:30.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:30.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:30.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:30.263665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:30.268122 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:30.268122 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:30.268122 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:30.268122 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 430/851] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:30.271253 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa19276-2873-4fe3-82a7-2a31ded4e249 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:30.299526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-d0f5c26d-4378-42cb-9b3d-8c261c4c975d None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:30.302334 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d0f5c26d-4378-42cb-9b3d-8c261c4c975d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:30.427998 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa19276-2873-4fe3-82a7-2a31ded4e249 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:30.437803 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 422/852] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:30 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:10:30.496440 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea222da-e8b7-4342-9c04-e1a9e0a08ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:30.497483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea222da-e8b7-4342-9c04-e1a9e0a08ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.497974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea222da-e8b7-4342-9c04-e1a9e0a08ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.498465 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea222da-e8b7-4342-9c04-e1a9e0a08ad9 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.500360 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea222da-e8b7-4342-9c04-e1a9e0a08ad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:30.500891 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea222da-e8b7-4342-9c04-e1a9e0a08ad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:30.503496 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea222da-e8b7-4342-9c04-e1a9e0a08ad9 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:30.527189 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 431/853] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:30.535777 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5c26d-4378-42cb-9b3d-8c261c4c975d None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:30.669812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5c26d-4378-42cb-9b3d-8c261c4c975d None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:30.682984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:30.712686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b019085-693b-4782-99da-04257df9d47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:30.713826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b019085-693b-4782-99da-04257df9d47a None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.714338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b019085-693b-4782-99da-04257df9d47a None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.714847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b019085-693b-4782-99da-04257df9d47a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.724069 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b019085-693b-4782-99da-04257df9d47a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:30.725933 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b019085-693b-4782-99da-04257df9d47a None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:30.739397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-7b019085-693b-4782-99da-04257df9d47a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:30.739397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:30.739397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:30.739397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:30.739397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:30.739397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:30.739397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:30.739397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:30.739397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:30.739397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:30.739397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:30.739397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:30.739397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:30.739397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:30.739397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:30.740444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:30.741364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:30.741364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:30.741364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:30.741364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:30.741364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:30.741364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:30.741364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:30.741364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:30.743779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 423/854] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:30.750401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea222da-e8b7-4342-9c04-e1a9e0a08ad9 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:30.786272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:30.881857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b019085-693b-4782-99da-04257df9d47a None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:30.926791 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea222da-e8b7-4342-9c04-e1a9e0a08ad9 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:30.989133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92549b6f-5e86-4fe2-a2da-3fe31da000d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:30.990145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92549b6f-5e86-4fe2-a2da-3fe31da000d6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.990678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92549b6f-5e86-4fe2-a2da-3fe31da000d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.991077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92549b6f-5e86-4fe2-a2da-3fe31da000d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.992198 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92549b6f-5e86-4fe2-a2da-3fe31da000d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:30.992743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92549b6f-5e86-4fe2-a2da-3fe31da000d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:30.995548 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92549b6f-5e86-4fe2-a2da-3fe31da000d6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:31.001555 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b019085-693b-4782-99da-04257df9d47a None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:31.022328 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 432/855] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:31.024160 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92549b6f-5e86-4fe2-a2da-3fe31da000d6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:31.070619 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a236492-7333-4527-9ca2-2ca4c8d8e44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:31.071587 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a236492-7333-4527-9ca2-2ca4c8d8e44d None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.072078 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a236492-7333-4527-9ca2-2ca4c8d8e44d None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.072546 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a236492-7333-4527-9ca2-2ca4c8d8e44d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.090263 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a236492-7333-4527-9ca2-2ca4c8d8e44d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:31.097182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-2a236492-7333-4527-9ca2-2ca4c8d8e44d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:31.097182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:31.097182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:31.097182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:31.097182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:31.097182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:31.097182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:31.097182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:31.097182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:31.097182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:31.097182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:31.097182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:31.097182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:31.097182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:31.097182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:31.097182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:31.099039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:31.099039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:31.099039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:31.099039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:31.099039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:31.099039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:31.099039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:31.099039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:31.099039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:31.099039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:31.099039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:31.099039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:31.099039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:31.099039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:31.099039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:31.099039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:31.100705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:31.100705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:31.100705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:31.100705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:31.100705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:31.100705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:31.100705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:31.100705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:31.100705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:31.100705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:31.100705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:31.100705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:31.100705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 424/856] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:31.112731 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a236492-7333-4527-9ca2-2ca4c8d8e44d None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:31.147765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:31.170150 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92549b6f-5e86-4fe2-a2da-3fe31da000d6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:31.172285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:31.211466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c3920-561d-4f18-a02d-e900b12f78fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:31.212024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c3920-561d-4f18-a02d-e900b12f78fa None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.212301 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c3920-561d-4f18-a02d-e900b12f78fa None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.212571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c3920-561d-4f18-a02d-e900b12f78fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.213491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c3920-561d-4f18-a02d-e900b12f78fa None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=601a3defca70428689ee22517d7bcb19) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:31.213799 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c3920-561d-4f18-a02d-e900b12f78fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:31.215636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c3920-561d-4f18-a02d-e900b12f78fa None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:31.229089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 433/857] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:31 2022] GET /identity/v3/role_assignments?user.id=601a3defca70428689ee22517d7bcb19&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=dc5d71ab685e420697f520aa3a368177 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:31.285477 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a236492-7333-4527-9ca2-2ca4c8d8e44d None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:31.349079 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffa696-6559-4621-9a21-a8abe7aae55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:31.349964 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffa696-6559-4621-9a21-a8abe7aae55e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.350263 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffa696-6559-4621-9a21-a8abe7aae55e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.350567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffa696-6559-4621-9a21-a8abe7aae55e None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.352004 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffa696-6559-4621-9a21-a8abe7aae55e None admin] RBAC: Adding query filter params (name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:31.352577 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffa696-6559-4621-9a21-a8abe7aae55e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:31.355612 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffa696-6559-4621-9a21-a8abe7aae55e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:31.374960 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 425/858] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:31.384544 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c3920-561d-4f18-a02d-e900b12f78fa None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:31.530213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c3920-561d-4f18-a02d-e900b12f78fa None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:31.582140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94edc5e-d721-4a91-b630-4038b5e1d2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:31.582823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94edc5e-d721-4a91-b630-4038b5e1d2ff None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.583149 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94edc5e-d721-4a91-b630-4038b5e1d2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.583420 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94edc5e-d721-4a91-b630-4038b5e1d2ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.596859 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94edc5e-d721-4a91-b630-4038b5e1d2ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:31.598374 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94edc5e-d721-4a91-b630-4038b5e1d2ff None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:31.608916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:31.617415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-d94edc5e-d721-4a91-b630-4038b5e1d2ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:31.617415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:31.617415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:31.617415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:31.617415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:31.617415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:31.617415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:31.617415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:31.617415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:31.617415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:31.617415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:31.617415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:31.617415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:31.617415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:31.617415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:31.617415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:31.618422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:31.619310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:31.619310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:31.619310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:31.619310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 434/859] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:31.630253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffa696-6559-4621-9a21-a8abe7aae55e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:31.650190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3463a0-5a5a-408c-8293-4f29b21eb1b2 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.650520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3463a0-5a5a-408c-8293-4f29b21eb1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.650741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3463a0-5a5a-408c-8293-4f29b21eb1b2 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.652460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 435/860] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:31.658504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217010c-e683-44c6-b190-f204281eb6ef None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.658929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217010c-e683-44c6-b190-f204281eb6ef None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.659086 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217010c-e683-44c6-b190-f204281eb6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.681623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6217010c-e683-44c6-b190-f204281eb6ef None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:31.683230 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6217010c-e683-44c6-b190-f204281eb6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:31.773422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 436/861] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:31 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:10:31.794596 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:31.806419 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffa696-6559-4621-9a21-a8abe7aae55e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:31.848877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f2116-b357-419a-b472-5f5b980fe616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:31.849532 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f2116-b357-419a-b472-5f5b980fe616 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.849811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f2116-b357-419a-b472-5f5b980fe616 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.850167 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f2116-b357-419a-b472-5f5b980fe616 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.850890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f2116-b357-419a-b472-5f5b980fe616 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:31.851190 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f2116-b357-419a-b472-5f5b980fe616 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:31.852625 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f2116-b357-419a-b472-5f5b980fe616 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:31.868524 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 426/862] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:31.876875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217010c-e683-44c6-b190-f204281eb6ef None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:32.005885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217010c-e683-44c6-b190-f204281eb6ef None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:32.039412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:32.047035 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd007de-99ce-474f-ae07-530666a894aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:32.047730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd007de-99ce-474f-ae07-530666a894aa None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.048004 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd007de-99ce-474f-ae07-530666a894aa None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.048313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd007de-99ce-474f-ae07-530666a894aa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.057446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd007de-99ce-474f-ae07-530666a894aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:32.059052 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd007de-99ce-474f-ae07-530666a894aa None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:32.073686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-4cd007de-99ce-474f-ae07-530666a894aa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:10:32.073686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:32.073686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:32.073686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:32.073686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:32.073686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:32.073686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:32.073686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:32.073686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:32.073686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:32.073686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:32.073686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:32.073686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:32.073686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:32.073686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:32.073686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:32.075123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:32.076629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:32.076629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:32.076629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:32.076629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:32.076629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:32.076629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:32.076629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:32.076629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:32.076629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:10:32.076629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:32.076629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 437/863] 10.222.0.88 () {56 vars in 1117 bytes} [Tue Aug 23 22:10:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:32.082111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f2116-b357-419a-b472-5f5b980fe616 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:32.103001 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd007de-99ce-474f-ae07-530666a894aa None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:32.224644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd007de-99ce-474f-ae07-530666a894aa None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:32.268271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815bdf4-7019-41a5-9f56-815e47ce5c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:32.269426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815bdf4-7019-41a5-9f56-815e47ce5c2e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.269974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815bdf4-7019-41a5-9f56-815e47ce5c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.270442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815bdf4-7019-41a5-9f56-815e47ce5c2e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.271232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815bdf4-7019-41a5-9f56-815e47ce5c2e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:32.273069 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815bdf4-7019-41a5-9f56-815e47ce5c2e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:32.283617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-8815bdf4-7019-41a5-9f56-815e47ce5c2e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:10:32.283617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:32.283617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:32.283617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:32.283617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:32.283617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:32.283617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:32.283617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:32.283617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:10:32.283617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:32.283617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:32.283617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:32.283617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:32.283617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:32.283617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:32.283617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:32.284779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:32.286718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:32.286718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:32.286718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:32.286718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:32.286718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:32.286718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:32.286718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:32.286718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:32.286718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:32.286718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:32.286718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:32.286718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:32.286718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:10:32.286718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:32.293444 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 438/864] 10.222.0.88 () {56 vars in 1132 bytes} [Tue Aug 23 22:10:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:32.296778 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f2116-b357-419a-b472-5f5b980fe616 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:32.297430 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815bdf4-7019-41a5-9f56-815e47ce5c2e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:32.354051 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17997e5a-b8ba-483f-8827-b5008af7f3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:32.354547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17997e5a-b8ba-483f-8827-b5008af7f3eb None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.355082 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17997e5a-b8ba-483f-8827-b5008af7f3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.355519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17997e5a-b8ba-483f-8827-b5008af7f3eb None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.356896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17997e5a-b8ba-483f-8827-b5008af7f3eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:32.357770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17997e5a-b8ba-483f-8827-b5008af7f3eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:32.359940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17997e5a-b8ba-483f-8827-b5008af7f3eb None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:32.370256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:32.397091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 427/865] 10.222.0.88 () {56 vars in 1149 bytes} [Tue Aug 23 22:10:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:32.397091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791abcc0-af43-4295-b0fb-64c816cbb416 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.397091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791abcc0-af43-4295-b0fb-64c816cbb416 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.397091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791abcc0-af43-4295-b0fb-64c816cbb416 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.397091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 428/866] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:32.408935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763318d5-d347-4a7f-b47c-8489b0451ac9 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.409569 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763318d5-d347-4a7f-b47c-8489b0451ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.410055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763318d5-d347-4a7f-b47c-8489b0451ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.433012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-763318d5-d347-4a7f-b47c-8489b0451ac9 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:32.434791 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-763318d5-d347-4a7f-b47c-8489b0451ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:32.442183 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:32.449895 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815bdf4-7019-41a5-9f56-815e47ce5c2e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:32.506677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227661cb-eb3b-454e-9cbb-75e5d13d9787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:32.507380 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227661cb-eb3b-454e-9cbb-75e5d13d9787 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.507882 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227661cb-eb3b-454e-9cbb-75e5d13d9787 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.508368 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227661cb-eb3b-454e-9cbb-75e5d13d9787 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.509563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227661cb-eb3b-454e-9cbb-75e5d13d9787 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:32.510114 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227661cb-eb3b-454e-9cbb-75e5d13d9787 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:32.512589 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227661cb-eb3b-454e-9cbb-75e5d13d9787 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:32.521551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 439/867] 10.222.0.88 () {56 vars in 1126 bytes} [Tue Aug 23 22:10:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:32.522853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227661cb-eb3b-454e-9cbb-75e5d13d9787 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:32.540706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 429/868] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:32 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:32.542623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763318d5-d347-4a7f-b47c-8489b0451ac9 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:32.705020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227661cb-eb3b-454e-9cbb-75e5d13d9787 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:32.716282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763318d5-d347-4a7f-b47c-8489b0451ac9 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:32.761630 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:32.775753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ff17d-46de-457d-a66d-83f24f22a8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:32.776804 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ff17d-46de-457d-a66d-83f24f22a8bd None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.777219 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b89e2-25e7-47d6-9df8-b0c1f370516f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:32.777432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ff17d-46de-457d-a66d-83f24f22a8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.777871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ff17d-46de-457d-a66d-83f24f22a8bd None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.778125 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b89e2-25e7-47d6-9df8-b0c1f370516f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.778504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b89e2-25e7-47d6-9df8-b0c1f370516f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.778876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b89e2-25e7-47d6-9df8-b0c1f370516f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.779138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ff17d-46de-457d-a66d-83f24f22a8bd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:32.779590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ff17d-46de-457d-a66d-83f24f22a8bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:32.781909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ff17d-46de-457d-a66d-83f24f22a8bd None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:32.788997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 430/869] 10.222.0.88 () {56 vars in 1126 bytes} [Tue Aug 23 22:10:32 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:32.844713 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ff17d-46de-457d-a66d-83f24f22a8bd None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:32.849966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:32.918330 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b89e2-25e7-47d6-9df8-b0c1f370516f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:32.921161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b89e2-25e7-47d6-9df8-b0c1f370516f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:32.983527 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ff17d-46de-457d-a66d-83f24f22a8bd None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:33.049674 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754e15f-7db6-487b-90c8-137e179da126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:33.050598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754e15f-7db6-487b-90c8-137e179da126 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:33.051052 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754e15f-7db6-487b-90c8-137e179da126 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:33.051514 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754e15f-7db6-487b-90c8-137e179da126 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:33.052647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754e15f-7db6-487b-90c8-137e179da126 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:33.055049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754e15f-7db6-487b-90c8-137e179da126 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-2754e15f-7db6-487b-90c8-137e179da126 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:33.071937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:33.073858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:33.073858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:33.073858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:33.073858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:33.073858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:33.073858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:33.073858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:33.073858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:33.073858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:33.073858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:33.073858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:33.073858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:33.073858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:33.073858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:33.073858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:33.073858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:33.075526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:33.075526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:33.075526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:33.075526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:33.075526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:33.075526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:33.075526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:33.075526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:33.075526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:33.075526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:33.075526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:33.075526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:33.075526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:10:33.075526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:33.075526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 431/870] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:33.085616 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754e15f-7db6-487b-90c8-137e179da126 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:33.096453 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 440/871] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:32 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:10:33.106595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b89e2-25e7-47d6-9df8-b0c1f370516f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:33.144546 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:33.240283 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:33.318721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754e15f-7db6-487b-90c8-137e179da126 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:33.332408 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b89e2-25e7-47d6-9df8-b0c1f370516f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:33.396157 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0e96b-0c5e-42b3-a1a4-33a749ff5027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:33.397602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0e96b-0c5e-42b3-a1a4-33a749ff5027 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:33.399443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0e96b-0c5e-42b3-a1a4-33a749ff5027 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:33.400452 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0e96b-0c5e-42b3-a1a4-33a749ff5027 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:33.402315 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0e96b-0c5e-42b3-a1a4-33a749ff5027 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:33.403196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0e96b-0c5e-42b3-a1a4-33a749ff5027 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:33.407620 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0e96b-0c5e-42b3-a1a4-33a749ff5027 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:33.408753 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c2be0-21cb-4e11-8ab8-90c7321c6750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:33.409810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c2be0-21cb-4e11-8ab8-90c7321c6750 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:33.410279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c2be0-21cb-4e11-8ab8-90c7321c6750 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:33.410763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c2be0-21cb-4e11-8ab8-90c7321c6750 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:33.419444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 432/872] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:33.431333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0e96b-0c5e-42b3-a1a4-33a749ff5027 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:33.437524 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c2be0-21cb-4e11-8ab8-90c7321c6750 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:33.442502 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c2be0-21cb-4e11-8ab8-90c7321c6750 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:33.462538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-221c2be0-21cb-4e11-8ab8-90c7321c6750 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:33.462538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:33.462538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:33.462538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:33.462538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:33.462538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:33.462538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:33.462538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:33.462538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:33.462538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:33.462538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:33.462538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:33.462538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:33.462538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:33.462538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:33.462538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:33.464321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:33.466048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:33.466048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:33.466048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:33.466048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 441/873] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:33.475901 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c2be0-21cb-4e11-8ab8-90c7321c6750 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:33.572522 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0e96b-0c5e-42b3-a1a4-33a749ff5027 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:33.575522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:33.609758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:33.617815 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdb150-8f50-4f34-b1ef-ba14256f332d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:33.618955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdb150-8f50-4f34-b1ef-ba14256f332d None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:33.619390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdb150-8f50-4f34-b1ef-ba14256f332d None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:33.619812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdb150-8f50-4f34-b1ef-ba14256f332d None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:33.620261 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c2be0-21cb-4e11-8ab8-90c7321c6750 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:33.620817 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebdb150-8f50-4f34-b1ef-ba14256f332d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:33.621303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebdb150-8f50-4f34-b1ef-ba14256f332d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:33.624686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebdb150-8f50-4f34-b1ef-ba14256f332d None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:33.631410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 433/874] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:33.635143 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71508338-86f8-4862-932c-fc6f13e6c500 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:33.635584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71508338-86f8-4862-932c-fc6f13e6c500 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:33.635945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71508338-86f8-4862-932c-fc6f13e6c500 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:33.637888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 434/875] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:33.638825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71508338-86f8-4862-932c-fc6f13e6c500 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:33.667336 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b8b4d-3ab3-4518-bcac-ca37d5d58c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:33.668254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b8b4d-3ab3-4518-bcac-ca37d5d58c6d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:33.668745 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b8b4d-3ab3-4518-bcac-ca37d5d58c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:33.669018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b8b4d-3ab3-4518-bcac-ca37d5d58c6d None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:33.669699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b8b4d-3ab3-4518-bcac-ca37d5d58c6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:33.669999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b8b4d-3ab3-4518-bcac-ca37d5d58c6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:33.671436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b8b4d-3ab3-4518-bcac-ca37d5d58c6d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:33.685324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 442/876] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:33.686900 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577531b1-79e8-49d2-a5fb-58f5b4de83c0 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:33.687438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577531b1-79e8-49d2-a5fb-58f5b4de83c0 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:33.687708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577531b1-79e8-49d2-a5fb-58f5b4de83c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:33.708506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-577531b1-79e8-49d2-a5fb-58f5b4de83c0 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:33.709790 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-577531b1-79e8-49d2-a5fb-58f5b4de83c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:33.781958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71508338-86f8-4862-932c-fc6f13e6c500 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:33.797246 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 443/877] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:33 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:33.797246 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577531b1-79e8-49d2-a5fb-58f5b4de83c0 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:33.830290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c110c6-3032-4db3-9d57-2d0f814dcdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:33.830795 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c110c6-3032-4db3-9d57-2d0f814dcdac None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:33.831131 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c110c6-3032-4db3-9d57-2d0f814dcdac None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:33.831397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c110c6-3032-4db3-9d57-2d0f814dcdac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:33.832124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c110c6-3032-4db3-9d57-2d0f814dcdac None admin] RBAC: Adding query filter params (interface=public, service_id=b41990373fd7460aa2bcb6ad309a15cc, region_id=RegionOne) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:33.832429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c110c6-3032-4db3-9d57-2d0f814dcdac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:33.833869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c110c6-3032-4db3-9d57-2d0f814dcdac None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:33.849510 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 435/878] 10.222.0.88 () {56 vars in 1251 bytes} [Tue Aug 23 22:10:33 2022] GET /identity/v3/endpoints?service_id=b41990373fd7460aa2bcb6ad309a15cc&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:33.851780 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c110c6-3032-4db3-9d57-2d0f814dcdac None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:33.925173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:33.971079 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:34.000431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577531b1-79e8-49d2-a5fb-58f5b4de83c0 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:34.051864 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c110c6-3032-4db3-9d57-2d0f814dcdac None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:34.057334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00995cd-36d1-4329-8d13-299ec2b5ceac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:34.058034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00995cd-36d1-4329-8d13-299ec2b5ceac None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:34.058316 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00995cd-36d1-4329-8d13-299ec2b5ceac None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:34.058590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00995cd-36d1-4329-8d13-299ec2b5ceac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:34.076234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00995cd-36d1-4329-8d13-299ec2b5ceac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:34.080996 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00995cd-36d1-4329-8d13-299ec2b5ceac None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:34.101122 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda2d5d-c7b2-49ea-b0da-8ecaa1517013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:34.101937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda2d5d-c7b2-49ea-b0da-8ecaa1517013 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:34.102113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda2d5d-c7b2-49ea-b0da-8ecaa1517013 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:34.102481 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda2d5d-c7b2-49ea-b0da-8ecaa1517013 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:34.102946 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda2d5d-c7b2-49ea-b0da-8ecaa1517013 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:34.104477 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda2d5d-c7b2-49ea-b0da-8ecaa1517013 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:34.112226 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 436/879] 10.222.0.88 () {56 vars in 1087 bytes} [Tue Aug 23 22:10:33 2022] GET /identity/v3/services => generated 1011 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:10:34.113326 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda2d5d-c7b2-49ea-b0da-8ecaa1517013 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:34.115139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-c00995cd-36d1-4329-8d13-299ec2b5ceac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:34.115139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:34.115139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:34.115139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:34.115139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:34.115139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:34.115139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:34.115139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:34.115139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:34.115139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:34.115139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:34.115139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:34.115139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:34.115139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:34.115139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:34.116875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:34.118668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:34.118668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:34.118668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:34.118668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:34.118668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:34.118668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:34.118668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:34.118668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:34.123120 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 444/880] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:34.133121 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00995cd-36d1-4329-8d13-299ec2b5ceac None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:34.284232 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda2d5d-c7b2-49ea-b0da-8ecaa1517013 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:34.286603 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00995cd-36d1-4329-8d13-299ec2b5ceac None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:34.318499 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:34.325324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:34.343603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7501e-c01d-4c32-81cc-233ef7bfb4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:34.343810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7501e-c01d-4c32-81cc-233ef7bfb4ef None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:34.344206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7501e-c01d-4c32-81cc-233ef7bfb4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:34.344651 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7501e-c01d-4c32-81cc-233ef7bfb4ef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:34.345581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c7501e-c01d-4c32-81cc-233ef7bfb4ef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:34.347391 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c7501e-c01d-4c32-81cc-233ef7bfb4ef None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-79c7501e-c01d-4c32-81cc-233ef7bfb4ef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:34.358415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:34.360423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:34.360423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:34.360423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:34.360423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:34.360423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:34.360423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:34.360423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:34.360423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:34.360423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:34.360423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:34.360423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:34.360423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:34.360423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:34.360423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:34.360423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:34.360423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:34.361844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:34.361844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:34.361844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:34.361844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:34.361844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:34.361844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:34.361844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:34.361844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:34.361844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:34.361844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:34.361844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:34.361844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:34.361844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:10:34.361844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:34.361844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 437/881] 10.222.0.88 () {56 vars in 1126 bytes} [Tue Aug 23 22:10:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:34.365629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f87c85-c62f-4b8b-8557-0c19bd79bf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:34.366021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7501e-c01d-4c32-81cc-233ef7bfb4ef None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:34.366590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f87c85-c62f-4b8b-8557-0c19bd79bf88 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:34.367783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f87c85-c62f-4b8b-8557-0c19bd79bf88 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:34.367783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f87c85-c62f-4b8b-8557-0c19bd79bf88 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:34.368584 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f87c85-c62f-4b8b-8557-0c19bd79bf88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:34.369038 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f87c85-c62f-4b8b-8557-0c19bd79bf88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:34.371754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f87c85-c62f-4b8b-8557-0c19bd79bf88 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:34.392750 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 445/882] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:34.401200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f87c85-c62f-4b8b-8557-0c19bd79bf88 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:34.522342 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7501e-c01d-4c32-81cc-233ef7bfb4ef None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:34.578084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f87c85-c62f-4b8b-8557-0c19bd79bf88 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:34.597308 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3421d-f1b6-449e-9b69-d172cf0e58cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:34.598711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3421d-f1b6-449e-9b69-d172cf0e58cf None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:34.599276 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3421d-f1b6-449e-9b69-d172cf0e58cf None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:34.599846 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3421d-f1b6-449e-9b69-d172cf0e58cf None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:34.601562 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3421d-f1b6-449e-9b69-d172cf0e58cf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:34.602830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3421d-f1b6-449e-9b69-d172cf0e58cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:34.606457 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3421d-f1b6-449e-9b69-d172cf0e58cf None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:34.616807 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 438/883] 10.222.0.88 () {56 vars in 1122 bytes} [Tue Aug 23 22:10:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:34.624503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3421d-f1b6-449e-9b69-d172cf0e58cf None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:34.637685 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200b50d-389e-4512-a1c8-f8869772d31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:34.638307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200b50d-389e-4512-a1c8-f8869772d31b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:34.638590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200b50d-389e-4512-a1c8-f8869772d31b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:34.638864 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200b50d-389e-4512-a1c8-f8869772d31b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:34.640165 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200b50d-389e-4512-a1c8-f8869772d31b None admin] RBAC: Adding query filter params (role.id=23ea214daa734346a6da3a0a8c50d23a, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=cb001b14eec54047aee42ca6bd4a1eba) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:34.640477 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200b50d-389e-4512-a1c8-f8869772d31b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:34.641953 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200b50d-389e-4512-a1c8-f8869772d31b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:34.657232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 446/884] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:34 2022] GET /identity/v3/role_assignments?user.id=cb001b14eec54047aee42ca6bd4a1eba&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=23ea214daa734346a6da3a0a8c50d23a => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:34.738238 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:34.816620 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3421d-f1b6-449e-9b69-d172cf0e58cf None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:34.885077 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d0536-6ffb-4b7b-856a-88abaf51f7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:34.886012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d0536-6ffb-4b7b-856a-88abaf51f7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:34.886496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d0536-6ffb-4b7b-856a-88abaf51f7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:34.886957 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d0536-6ffb-4b7b-856a-88abaf51f7b0 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:34.888036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d0536-6ffb-4b7b-856a-88abaf51f7b0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:34.888565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d0536-6ffb-4b7b-856a-88abaf51f7b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:34.891798 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d0536-6ffb-4b7b-856a-88abaf51f7b0 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:34.901079 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 439/885] 10.222.0.88 () {56 vars in 1122 bytes} [Tue Aug 23 22:10:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:35.307315 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747a44c-c618-4ab1-97a7-c4fb69d27773 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:35.308096 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747a44c-c618-4ab1-97a7-c4fb69d27773 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:35.308096 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747a44c-c618-4ab1-97a7-c4fb69d27773 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:35.309488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 447/886] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:35.317131 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e82dd-8f5d-41a6-8203-2f6659ccf679 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:35.318104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e82dd-8f5d-41a6-8203-2f6659ccf679 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:35.318104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e82dd-8f5d-41a6-8203-2f6659ccf679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:35.348780 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-1f8e82dd-8f5d-41a6-8203-2f6659ccf679 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:35.350902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-1f8e82dd-8f5d-41a6-8203-2f6659ccf679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:35.443226 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 440/887] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:35 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:35.924736 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747a44c-c618-4ab1-97a7-c4fb69d27773 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:36.011377 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:36.121756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747a44c-c618-4ab1-97a7-c4fb69d27773 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:36.177313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5498da-c99b-412f-8864-d3513137dd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:36.177988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5498da-c99b-412f-8864-d3513137dd5e None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:36.178304 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5498da-c99b-412f-8864-d3513137dd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:36.178733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5498da-c99b-412f-8864-d3513137dd5e None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:36.179747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5498da-c99b-412f-8864-d3513137dd5e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:36.181185 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5498da-c99b-412f-8864-d3513137dd5e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:36.201479 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 448/888] 10.222.0.88 () {60 vars in 1141 bytes} [Tue Aug 23 22:10:35 2022] POST /identity/v3/services => generated 269 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:37.027666 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8c016-0d3b-4bfb-b18c-348c429db60d None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.029232 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8c016-0d3b-4bfb-b18c-348c429db60d None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.030568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8c016-0d3b-4bfb-b18c-348c429db60d None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.033857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 441/889] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:37.043292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86110c6-09ce-414c-b6c9-4964699c5b78 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.044745 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86110c6-09ce-414c-b6c9-4964699c5b78 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.046145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86110c6-09ce-414c-b6c9-4964699c5b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.078600 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d86110c6-09ce-414c-b6c9-4964699c5b78 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:37.081018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d86110c6-09ce-414c-b6c9-4964699c5b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:37.126100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d0695-be4c-401a-8048-adb253641352 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.126548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d0695-be4c-401a-8048-adb253641352 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.126732 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d0695-be4c-401a-8048-adb253641352 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.128376 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 442/890] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:37.134374 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214f64a-7800-4027-8ad5-c798f6020932 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.134794 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214f64a-7800-4027-8ad5-c798f6020932 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.134948 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214f64a-7800-4027-8ad5-c798f6020932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.156652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-b214f64a-7800-4027-8ad5-c798f6020932 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:37.158059 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-b214f64a-7800-4027-8ad5-c798f6020932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:37.201624 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 449/891] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:37.230253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:37.256806 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20aa949-2b25-4a2f-8c82-62447d8fcc57 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.257162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20aa949-2b25-4a2f-8c82-62447d8fcc57 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.257441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20aa949-2b25-4a2f-8c82-62447d8fcc57 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.259189 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 450/892] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:37.264591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a08f0-32d4-4908-945c-1e143efb6a8c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.264884 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a08f0-32d4-4908-945c-1e143efb6a8c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.265217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a08f0-32d4-4908-945c-1e143efb6a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.281230 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 443/893] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:37.284894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7f7a08f0-32d4-4908-945c-1e143efb6a8c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:37.286161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7f7a08f0-32d4-4908-945c-1e143efb6a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:37.313632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:37.360739 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 451/894] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:10:37.485736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214f64a-7800-4027-8ad5-c798f6020932 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:37.651815 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a08f0-32d4-4908-945c-1e143efb6a8c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:37.673079 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214f64a-7800-4027-8ad5-c798f6020932 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:37.741184 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31613d2b-5412-49dc-a871-611117e4b501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:37.742296 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31613d2b-5412-49dc-a871-611117e4b501 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.742946 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31613d2b-5412-49dc-a871-611117e4b501 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.743659 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31613d2b-5412-49dc-a871-611117e4b501 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.744761 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31613d2b-5412-49dc-a871-611117e4b501 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:37.747437 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31613d2b-5412-49dc-a871-611117e4b501 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-31613d2b-5412-49dc-a871-611117e4b501 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:37.767930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:37.769783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:37.769783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:37.769783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:37.769783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:37.769783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:37.769783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:37.769783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:37.769783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:37.769783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:37.769783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:37.769783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:37.769783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:37.769783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:37.769783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:37.769783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:37.769783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:37.770635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:37.770635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:37.770635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:37.770635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:37.770635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:37.770635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:37.770635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:37.770635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:37.770635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:37.770635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:37.770635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:37.770635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:37.770635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:10:37.770635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:37.770635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 444/895] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:37.771969 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31613d2b-5412-49dc-a871-611117e4b501 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:37.781341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a08f0-32d4-4908-945c-1e143efb6a8c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:37.824748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873c8de-fa75-4b4b-927a-a9e772053d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:37.825741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873c8de-fa75-4b4b-927a-a9e772053d82 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.826173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873c8de-fa75-4b4b-927a-a9e772053d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.826602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873c8de-fa75-4b4b-927a-a9e772053d82 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.842728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873c8de-fa75-4b4b-927a-a9e772053d82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:37.848214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:37.853980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-3873c8de-fa75-4b4b-927a-a9e772053d82 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:10:37.853980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:37.853980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:37.853980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:37.853980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:37.853980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:37.853980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:37.853980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:37.853980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:37.853980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:37.853980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:37.853980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:37.853980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:37.853980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:37.853980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:37.853980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:37.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:37.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:37.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:37.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:37.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:37.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:37.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:37.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:37.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:37.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:37.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:37.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:37.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:37.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:37.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:37.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:37.855941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:37.855941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:37.855941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:37.855941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:37.855941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:37.855941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:37.855941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:37.855941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:37.855941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:37.855941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:37.855941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:10:37.855941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:37.856639 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 452/896] 10.222.0.88 () {56 vars in 1120 bytes} [Tue Aug 23 22:10:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:37.857481 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873c8de-fa75-4b4b-927a-a9e772053d82 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:37.891463 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:37.942120 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31613d2b-5412-49dc-a871-611117e4b501 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:38.012110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0b694-97a0-4dbb-8e7c-c2c0b93c2133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:38.013416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0b694-97a0-4dbb-8e7c-c2c0b93c2133 None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.013784 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0b694-97a0-4dbb-8e7c-c2c0b93c2133 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.014254 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0b694-97a0-4dbb-8e7c-c2c0b93c2133 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.015420 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf0b694-97a0-4dbb-8e7c-c2c0b93c2133 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:38.018612 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf0b694-97a0-4dbb-8e7c-c2c0b93c2133 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:38.033483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873c8de-fa75-4b4b-927a-a9e772053d82 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:38.057273 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 445/897] 10.222.0.88 () {60 vars in 1141 bytes} [Tue Aug 23 22:10:37 2022] POST /identity/v3/services => generated 241 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:38.057273 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0b694-97a0-4dbb-8e7c-c2c0b93c2133 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:38.139570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e9fbe-1359-403a-b6e5-4c27953f9166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:38.139570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e9fbe-1359-403a-b6e5-4c27953f9166 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.139570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e9fbe-1359-403a-b6e5-4c27953f9166 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.139570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e9fbe-1359-403a-b6e5-4c27953f9166 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.140840 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e9fbe-1359-403a-b6e5-4c27953f9166 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:38.140840 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e9fbe-1359-403a-b6e5-4c27953f9166 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:38.141545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e9fbe-1359-403a-b6e5-4c27953f9166 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:38.160692 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 453/898] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:38.164895 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e9fbe-1359-403a-b6e5-4c27953f9166 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:38.294202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:38.300792 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0b694-97a0-4dbb-8e7c-c2c0b93c2133 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:38.367566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:38.387458 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990657d-9965-4af5-b234-457b325dd3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:38.389386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990657d-9965-4af5-b234-457b325dd3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.390344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990657d-9965-4af5-b234-457b325dd3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.391110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990657d-9965-4af5-b234-457b325dd3c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.394960 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990657d-9965-4af5-b234-457b325dd3c9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:38.395878 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990657d-9965-4af5-b234-457b325dd3c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:38.405219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990657d-9965-4af5-b234-457b325dd3c9 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:38.414669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e9fbe-1359-403a-b6e5-4c27953f9166 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:38.429193 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 446/899] 10.222.0.88 () {56 vars in 1115 bytes} [Tue Aug 23 22:10:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:38.437439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990657d-9965-4af5-b234-457b325dd3c9 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:38.486753 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8489b-e1c6-436c-b891-f875de2d9d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:38.487865 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8489b-e1c6-436c-b891-f875de2d9d80 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.488409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8489b-e1c6-436c-b891-f875de2d9d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.488949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8489b-e1c6-436c-b891-f875de2d9d80 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.490181 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8489b-e1c6-436c-b891-f875de2d9d80 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:38.491486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8489b-e1c6-436c-b891-f875de2d9d80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:38.495849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8489b-e1c6-436c-b891-f875de2d9d80 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:38.508100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 454/900] 10.222.0.88 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:38.516738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8489b-e1c6-436c-b891-f875de2d9d80 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:38.653776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990657d-9965-4af5-b234-457b325dd3c9 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:38.712888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b974da-f56e-4e79-9ac2-59ddee208390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:38.713977 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b974da-f56e-4e79-9ac2-59ddee208390 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.714464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b974da-f56e-4e79-9ac2-59ddee208390 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.714949 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b974da-f56e-4e79-9ac2-59ddee208390 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.736460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b974da-f56e-4e79-9ac2-59ddee208390 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:38.739211 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b974da-f56e-4e79-9ac2-59ddee208390 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:38.767705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-a3b974da-f56e-4e79-9ac2-59ddee208390 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:38.767705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:38.767705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:38.767705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:38.767705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:38.767705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:38.767705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:38.767705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:38.767705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:38.767705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:38.767705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:38.767705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:38.767705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:38.767705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:38.767705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:38.767705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:38.768844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:38.769831 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:38.769831 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:38.769831 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:38.770540 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 447/901] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:38.778368 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b974da-f56e-4e79-9ac2-59ddee208390 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:38.791179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8489b-e1c6-436c-b891-f875de2d9d80 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:38.801794 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:38.840116 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48315ae1-342e-4041-ac00-2869219f25f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:38.840730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48315ae1-342e-4041-ac00-2869219f25f4 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.841045 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48315ae1-342e-4041-ac00-2869219f25f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.841335 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48315ae1-342e-4041-ac00-2869219f25f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.842111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48315ae1-342e-4041-ac00-2869219f25f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:38.843908 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48315ae1-342e-4041-ac00-2869219f25f4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:38.865122 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:38.875210 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 455/902] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:10:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:38.927922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b974da-f56e-4e79-9ac2-59ddee208390 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:38.980643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2354fd1-f253-4a65-abd0-9262f1051afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:38.981343 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2354fd1-f253-4a65-abd0-9262f1051afc None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.981620 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2354fd1-f253-4a65-abd0-9262f1051afc None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.981888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2354fd1-f253-4a65-abd0-9262f1051afc None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.982552 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2354fd1-f253-4a65-abd0-9262f1051afc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:38.982883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2354fd1-f253-4a65-abd0-9262f1051afc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:38.984462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2354fd1-f253-4a65-abd0-9262f1051afc None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:39.000867 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 448/903] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:39.010307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48315ae1-342e-4041-ac00-2869219f25f4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:39.146004 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48315ae1-342e-4041-ac00-2869219f25f4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:39.184180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536ccf3-5366-4c58-a549-c3f6d76bba00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:39.184812 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536ccf3-5366-4c58-a549-c3f6d76bba00 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:39.185120 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536ccf3-5366-4c58-a549-c3f6d76bba00 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:39.185393 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536ccf3-5366-4c58-a549-c3f6d76bba00 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:39.195755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536ccf3-5366-4c58-a549-c3f6d76bba00 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:39.197273 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536ccf3-5366-4c58-a549-c3f6d76bba00 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:39.212009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-b536ccf3-5366-4c58-a549-c3f6d76bba00 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:10:39.212009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:39.212009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:39.212009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:39.212009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:39.212009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:39.212009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:39.212009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:39.212009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:39.212009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:39.212009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:39.212009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:39.212009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:39.212009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:39.212009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:39.212009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:39.213024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:39.214279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:39.214279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:39.214279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:39.214279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:39.214279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:39.214279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:39.214279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:39.214279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:39.214279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:10:39.214279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:39.214279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 456/904] 10.222.0.88 () {56 vars in 1117 bytes} [Tue Aug 23 22:10:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:39.222362 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2354fd1-f253-4a65-abd0-9262f1051afc None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:39.264484 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1d59a-f0fe-4cf2-b764-8ac96150a2c5 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:39.264995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1d59a-f0fe-4cf2-b764-8ac96150a2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:39.265545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1d59a-f0fe-4cf2-b764-8ac96150a2c5 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:39.268480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 457/905] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:39.277498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914de81c-f083-4d33-94df-20d1119f5567 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:39.277971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914de81c-f083-4d33-94df-20d1119f5567 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:39.278452 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914de81c-f083-4d33-94df-20d1119f5567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:39.305670 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:39.320153 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-914de81c-f083-4d33-94df-20d1119f5567 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:39.323153 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-914de81c-f083-4d33-94df-20d1119f5567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:39.345374 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2354fd1-f253-4a65-abd0-9262f1051afc None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:39.396914 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d772fb9-8c99-4be3-8849-2b5dc5de2564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:39.397729 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d772fb9-8c99-4be3-8849-2b5dc5de2564 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:39.398215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d772fb9-8c99-4be3-8849-2b5dc5de2564 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:39.398700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d772fb9-8c99-4be3-8849-2b5dc5de2564 None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:39.400266 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d772fb9-8c99-4be3-8849-2b5dc5de2564 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:39.400884 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d772fb9-8c99-4be3-8849-2b5dc5de2564 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:39.403255 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d772fb9-8c99-4be3-8849-2b5dc5de2564 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:39.435199 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 449/906] 10.222.0.88 () {56 vars in 1149 bytes} [Tue Aug 23 22:10:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:39.443075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d772fb9-8c99-4be3-8849-2b5dc5de2564 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:39.450782 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:39.475842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 458/907] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:39.571897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d772fb9-8c99-4be3-8849-2b5dc5de2564 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:39.604870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:39.614169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c7a8e-270c-4c03-b9e6-28a149e8d6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:39.615292 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c7a8e-270c-4c03-b9e6-28a149e8d6bc None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:39.615566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c7a8e-270c-4c03-b9e6-28a149e8d6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:39.615859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c7a8e-270c-4c03-b9e6-28a149e8d6bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:39.698720 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c7a8e-270c-4c03-b9e6-28a149e8d6bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:39.700069 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c7a8e-270c-4c03-b9e6-28a149e8d6bc None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:39.814962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 450/908] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:39 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:10:39.820913 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914de81c-f083-4d33-94df-20d1119f5567 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:39.871798 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c7a8e-270c-4c03-b9e6-28a149e8d6bc None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:39.905520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:39.981351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914de81c-f083-4d33-94df-20d1119f5567 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.023519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7a6c6-a6e6-4650-a131-b3f29a78e282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.024295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7a6c6-a6e6-4650-a131-b3f29a78e282 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.024578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7a6c6-a6e6-4650-a131-b3f29a78e282 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.024850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7a6c6-a6e6-4650-a131-b3f29a78e282 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.037968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7a6c6-a6e6-4650-a131-b3f29a78e282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.039604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7a6c6-a6e6-4650-a131-b3f29a78e282 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.056429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c7a8e-270c-4c03-b9e6-28a149e8d6bc None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.057107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-1af7a6c6-a6e6-4650-a131-b3f29a78e282 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:40.057107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:40.057107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:40.057107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:40.057107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:40.057107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:40.057107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:40.057107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:40.057107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:40.057107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:40.057107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:40.057107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:40.057107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:40.057107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:40.057107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:40.057107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:40.058067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:40.059289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:40.059289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:40.059289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:40.059289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 459/909] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:40.066762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7a6c6-a6e6-4650-a131-b3f29a78e282 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.120541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:40.122863 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257f958-3dd4-41bf-91bd-8329622146d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.124256 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257f958-3dd4-41bf-91bd-8329622146d1 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.124818 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257f958-3dd4-41bf-91bd-8329622146d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.125363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257f958-3dd4-41bf-91bd-8329622146d1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.126507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257f958-3dd4-41bf-91bd-8329622146d1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.129455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257f958-3dd4-41bf-91bd-8329622146d1 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.144894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-d257f958-3dd4-41bf-91bd-8329622146d1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:10:40.144894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:40.144894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:40.144894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:40.144894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:40.144894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:40.144894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:40.144894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:40.144894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:10:40.144894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:40.144894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:40.144894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:40.144894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:40.144894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:40.144894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:40.144894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:40.145985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:40.147278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:40.147278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:40.147278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:40.147278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:40.147278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:40.147278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:40.147278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:40.147278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:40.147278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:40.147278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:40.147278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:40.147278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:40.147278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:10:40.147278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:40.147278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 451/910] 10.222.0.88 () {56 vars in 1132 bytes} [Tue Aug 23 22:10:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:40.153627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257f958-3dd4-41bf-91bd-8329622146d1 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.208300 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7a6c6-a6e6-4650-a131-b3f29a78e282 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.263289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a150c2-3c5a-4811-ac8a-220102c51389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.265440 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a150c2-3c5a-4811-ac8a-220102c51389 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.266340 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a150c2-3c5a-4811-ac8a-220102c51389 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.267019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a150c2-3c5a-4811-ac8a-220102c51389 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.268552 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a150c2-3c5a-4811-ac8a-220102c51389 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:40.269483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a150c2-3c5a-4811-ac8a-220102c51389 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.272383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a150c2-3c5a-4811-ac8a-220102c51389 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.281005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:40.291272 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257f958-3dd4-41bf-91bd-8329622146d1 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.305482 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 460/911] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:40.314704 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a150c2-3c5a-4811-ac8a-220102c51389 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.349084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fc4cb-29d5-4220-b8c6-3ee7354e77ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.349999 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fc4cb-29d5-4220-b8c6-3ee7354e77ae None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.350583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fc4cb-29d5-4220-b8c6-3ee7354e77ae None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.351089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fc4cb-29d5-4220-b8c6-3ee7354e77ae None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.352776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fc4cb-29d5-4220-b8c6-3ee7354e77ae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:40.353287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fc4cb-29d5-4220-b8c6-3ee7354e77ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.354700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fc4cb-29d5-4220-b8c6-3ee7354e77ae None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.362554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 452/912] 10.222.0.88 () {56 vars in 1126 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:40.370639 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fc4cb-29d5-4220-b8c6-3ee7354e77ae None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.474141 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a150c2-3c5a-4811-ac8a-220102c51389 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.476835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:40.529971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746e021-3fe0-44b1-a22b-f122031470ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.531073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746e021-3fe0-44b1-a22b-f122031470ae None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.531589 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746e021-3fe0-44b1-a22b-f122031470ae None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.532086 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746e021-3fe0-44b1-a22b-f122031470ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.545927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746e021-3fe0-44b1-a22b-f122031470ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.548720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746e021-3fe0-44b1-a22b-f122031470ae None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.551521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fc4cb-29d5-4220-b8c6-3ee7354e77ae None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.578664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-c746e021-3fe0-44b1-a22b-f122031470ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:40.578664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:40.578664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:40.578664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:40.578664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:40.578664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:40.578664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:40.578664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:40.578664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:40.578664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:40.578664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:40.578664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:40.578664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:40.578664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:40.578664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:40.579644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:40.580530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:40.580530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:40.580530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:40.580530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:40.580530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:40.580530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:40.580530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:40.580530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:40.586178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 461/913] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:40.591109 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746e021-3fe0-44b1-a22b-f122031470ae None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.600150 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be43a9-1a6d-47b6-a355-221bd2f6c821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.601198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be43a9-1a6d-47b6-a355-221bd2f6c821 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.601679 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be43a9-1a6d-47b6-a355-221bd2f6c821 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.602147 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be43a9-1a6d-47b6-a355-221bd2f6c821 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.602986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be43a9-1a6d-47b6-a355-221bd2f6c821 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:40.603502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be43a9-1a6d-47b6-a355-221bd2f6c821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.605110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be43a9-1a6d-47b6-a355-221bd2f6c821 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.612786 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 453/914] 10.222.0.88 () {56 vars in 1126 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:40.622169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be43a9-1a6d-47b6-a355-221bd2f6c821 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.652462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:40.750374 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be43a9-1a6d-47b6-a355-221bd2f6c821 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.790092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2db28-d385-4c7e-bc97-472c9904b519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.790600 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2db28-d385-4c7e-bc97-472c9904b519 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.790880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2db28-d385-4c7e-bc97-472c9904b519 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.791158 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2db28-d385-4c7e-bc97-472c9904b519 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.791869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2db28-d385-4c7e-bc97-472c9904b519 None admin] RBAC: Adding query filter params (interface=public, service_id=89862ad868794b43a56de935e4686c87, region_id=RegionOne) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:40.792179 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2db28-d385-4c7e-bc97-472c9904b519 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.793627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2db28-d385-4c7e-bc97-472c9904b519 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.800378 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 454/915] 10.222.0.88 () {56 vars in 1251 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/endpoints?service_id=89862ad868794b43a56de935e4686c87&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:40.806001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2db28-d385-4c7e-bc97-472c9904b519 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.816266 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746e021-3fe0-44b1-a22b-f122031470ae None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.890354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d75648-4f9e-4957-aa1b-6d2905c300a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.891963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d75648-4f9e-4957-aa1b-6d2905c300a0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.892920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d75648-4f9e-4957-aa1b-6d2905c300a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.893748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d75648-4f9e-4957-aa1b-6d2905c300a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.895532 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d75648-4f9e-4957-aa1b-6d2905c300a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:40.896427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d75648-4f9e-4957-aa1b-6d2905c300a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.899833 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d75648-4f9e-4957-aa1b-6d2905c300a0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.903567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:40.924033 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2db28-d385-4c7e-bc97-472c9904b519 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.936979 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 462/916] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:40.941480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d75648-4f9e-4957-aa1b-6d2905c300a0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.962920 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295746e-f851-4147-97af-1c457234c984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.963537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295746e-f851-4147-97af-1c457234c984 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.963813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295746e-f851-4147-97af-1c457234c984 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.964080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295746e-f851-4147-97af-1c457234c984 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.964686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295746e-f851-4147-97af-1c457234c984 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.966451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295746e-f851-4147-97af-1c457234c984 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.973818 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 455/917] 10.222.0.88 () {56 vars in 1087 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:10:40.997763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:41.150931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d75648-4f9e-4957-aa1b-6d2905c300a0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:41.192192 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564036c-25c2-4ecf-bcc2-b2593ce17435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:41.192822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564036c-25c2-4ecf-bcc2-b2593ce17435 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:41.193579 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564036c-25c2-4ecf-bcc2-b2593ce17435 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:41.193579 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564036c-25c2-4ecf-bcc2-b2593ce17435 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:41.194146 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564036c-25c2-4ecf-bcc2-b2593ce17435 None admin] RBAC: Adding query filter params (role.id=97c55e5f00404a29bb3f2df9566b6b0f, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=cb001b14eec54047aee42ca6bd4a1eba) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:41.194462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564036c-25c2-4ecf-bcc2-b2593ce17435 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:41.195928 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564036c-25c2-4ecf-bcc2-b2593ce17435 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:41.209186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 463/918] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/role_assignments?user.id=cb001b14eec54047aee42ca6bd4a1eba&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=97c55e5f00404a29bb3f2df9566b6b0f => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:41.297372 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10089e-896d-4c85-ab0c-38e108ddea0c None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:41.297902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10089e-896d-4c85-ab0c-38e108ddea0c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:41.298294 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10089e-896d-4c85-ab0c-38e108ddea0c None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:41.301283 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 456/919] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:41.311426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839560a9-3f55-40ac-b918-811a80915dfe None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:41.312165 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839560a9-3f55-40ac-b918-811a80915dfe None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:41.312669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839560a9-3f55-40ac-b918-811a80915dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:41.323937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65043ecc-79cc-4cfe-9feb-3de92db95b6a None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:41.324276 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65043ecc-79cc-4cfe-9feb-3de92db95b6a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:41.324497 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65043ecc-79cc-4cfe-9feb-3de92db95b6a None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:41.326345 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 457/920] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:41.335091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8d558-7e21-4910-98f4-9b7b3e42be96 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:41.335620 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8d558-7e21-4910-98f4-9b7b3e42be96 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:41.336089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8d558-7e21-4910-98f4-9b7b3e42be96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:41.341580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-839560a9-3f55-40ac-b918-811a80915dfe None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:41.342883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-839560a9-3f55-40ac-b918-811a80915dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:41.367320 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-85e8d558-7e21-4910-98f4-9b7b3e42be96 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:41.370653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-85e8d558-7e21-4910-98f4-9b7b3e42be96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:41.453339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 464/921] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Aug 23 22:10:41.511116 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 458/922] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:41.957289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839560a9-3f55-40ac-b918-811a80915dfe None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:41.976658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:42.009965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8d558-7e21-4910-98f4-9b7b3e42be96 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:42.084293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839560a9-3f55-40ac-b918-811a80915dfe None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:42.139840 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b48444-7080-4488-889b-732fe24a4be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:42.140606 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b48444-7080-4488-889b-732fe24a4be5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:42.141134 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b48444-7080-4488-889b-732fe24a4be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:42.141622 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b48444-7080-4488-889b-732fe24a4be5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:42.142854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b48444-7080-4488-889b-732fe24a4be5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:42.144414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:42.145544 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b48444-7080-4488-889b-732fe24a4be5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-65b48444-7080-4488-889b-732fe24a4be5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:42.162733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:42.164010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:42.164010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:42.164010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:42.164010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:42.164010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:42.164010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:42.164010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:42.164010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:42.164010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:42.164010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:42.164010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:42.164010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:42.164010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:42.164010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:42.164010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:42.164010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:42.164984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:42.164984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:42.164984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:42.164984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:42.164984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:42.164984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:42.164984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:42.164984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:42.164984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:42.164984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:42.164984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:42.164984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:42.164984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:10:42.164984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:42.165749 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 465/923] 10.222.0.88 () {56 vars in 1126 bytes} [Tue Aug 23 22:10:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:42.174466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b48444-7080-4488-889b-732fe24a4be5 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:42.183080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8d558-7e21-4910-98f4-9b7b3e42be96 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:42.231969 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e22109-ec22-4369-893a-c7f2aab48460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:42.233819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e22109-ec22-4369-893a-c7f2aab48460 None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:42.234257 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e22109-ec22-4369-893a-c7f2aab48460 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:42.235082 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e22109-ec22-4369-893a-c7f2aab48460 None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:42.235958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e22109-ec22-4369-893a-c7f2aab48460 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:42.238406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e22109-ec22-4369-893a-c7f2aab48460 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:42.241661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: WARNING keystone.server.flask.common [None req-e9e22109-ec22-4369-893a-c7f2aab48460 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 23 22:10:42.265828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.sql.core [None req-e9e22109-ec22-4369-893a-c7f2aab48460 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 22:10:42.265828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 23 22:10:42.265828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [parameters: {'id': 'adc99de87fc94c7cb12369202279c802', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 23 22:10:42.265828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81679) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 22:10:42.287852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-e9e22109-ec22-4369-893a-c7f2aab48460 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id adc99de87fc94c7cb12369202279c802.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id adc99de87fc94c7cb12369202279c802. Aug 23 22:10:42.287852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:42.287852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 22:10:42.287852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 22:10:42.287852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 22:10:42.287852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 22:10:42.287852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 22:10:42.287852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 22:10:42.287852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 22:10:42.287852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application conn.query(q) Aug 23 22:10:42.287852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 22:10:42.287852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 22:10:42.287852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application result.read() Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 23 22:10:42.288855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application next(self.gen) Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application yield resource Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application next(self.gen) Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise self.value Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application session.commit() Aug 23 22:10:42.290047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self.session.flush() Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application compat.raise_( Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 22:10:42.291871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 22:10:42.293212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 22:10:42.293212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 22:10:42.293212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 22:10:42.293212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 22:10:42.293212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 23 22:10:42.293212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 23 22:10:42.293212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 22:10:42.293212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 22:10:42.293212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 22:10:42.293212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 22:10:42.293212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 22:10:42.293212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 22:10:42.293212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 22:10:42.293212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 22:10:42.293212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application conn.query(q) Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application result.read() Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 22:10:42.294155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 22:10:42.295926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 22:10:42.295926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 22:10:42.295926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 22:10:42.295926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 22:10:42.295926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 22:10:42.295926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 22:10:42.295926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 22:10:42.295926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 23 22:10:42.295926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [parameters: {'id': 'adc99de87fc94c7cb12369202279c802', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 23 22:10:42.295926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 22:10:42.295926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:42.295926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:42.295926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:42.295926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:42.295926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:42.296965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:42.300741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:42.300741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 23 22:10:42.300741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 23 22:10:42.300741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:42.300741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:42.300741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 23 22:10:42.300741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 23 22:10:42.300741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id adc99de87fc94c7cb12369202279c802. Aug 23 22:10:42.300741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:42.302580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 459/924] 10.222.0.88 () {60 vars in 1140 bytes} [Tue Aug 23 22:10:42 2022] POST /identity/v3/projects => generated 262 bytes in 294 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:42.312882 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e22109-ec22-4369-893a-c7f2aab48460 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:42.368195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:42.411912 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b48444-7080-4488-889b-732fe24a4be5 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:42.436513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e22109-ec22-4369-893a-c7f2aab48460 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:42.476402 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89304d9c-7a5c-4f6d-a7ff-0cfb179b503e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:42.477100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89304d9c-7a5c-4f6d-a7ff-0cfb179b503e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:42.477417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89304d9c-7a5c-4f6d-a7ff-0cfb179b503e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:42.477639 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89304d9c-7a5c-4f6d-a7ff-0cfb179b503e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:42.480104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:42.484635 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ceb347-9ebd-4361-99d5-cb07d0dea39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:42.486134 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ceb347-9ebd-4361-99d5-cb07d0dea39e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:42.488100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ceb347-9ebd-4361-99d5-cb07d0dea39e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:42.488983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89304d9c-7a5c-4f6d-a7ff-0cfb179b503e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:42.489290 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ceb347-9ebd-4361-99d5-cb07d0dea39e None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:42.490716 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ceb347-9ebd-4361-99d5-cb07d0dea39e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:42.491387 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89304d9c-7a5c-4f6d-a7ff-0cfb179b503e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:42.491680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ceb347-9ebd-4361-99d5-cb07d0dea39e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:42.495068 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ceb347-9ebd-4361-99d5-cb07d0dea39e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:42.505971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-89304d9c-7a5c-4f6d-a7ff-0cfb179b503e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:42.505971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:42.505971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:42.505971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:42.505971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:42.505971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:42.505971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:42.505971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:42.505971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:42.505971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:42.505971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:42.505971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:42.505971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:42.505971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:42.505971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:42.507702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:42.509503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:42.509503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:42.509503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:42.509503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:42.509503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:42.509503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:42.509503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:42.509503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:42.510747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 466/925] 10.222.0.88 () {56 vars in 1122 bytes} [Tue Aug 23 22:10:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:42.512870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 460/926] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:42.515343 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ceb347-9ebd-4361-99d5-cb07d0dea39e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:42.520607 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89304d9c-7a5c-4f6d-a7ff-0cfb179b503e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:42.688609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89304d9c-7a5c-4f6d-a7ff-0cfb179b503e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:42.729860 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ceb347-9ebd-4361-99d5-cb07d0dea39e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:42.764059 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959c659-24f7-47c8-a066-0d4f40afa3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:42.765572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959c659-24f7-47c8-a066-0d4f40afa3af None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:42.766011 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959c659-24f7-47c8-a066-0d4f40afa3af None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:42.766452 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959c659-24f7-47c8-a066-0d4f40afa3af None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:42.767612 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959c659-24f7-47c8-a066-0d4f40afa3af None admin] RBAC: Adding query filter params (name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:42.768106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959c659-24f7-47c8-a066-0d4f40afa3af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:42.770685 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959c659-24f7-47c8-a066-0d4f40afa3af None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:42.818901 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502d830-240b-47ca-96e2-e595e2e3fde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:42.820381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502d830-240b-47ca-96e2-e595e2e3fde2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:42.821117 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 461/927] 10.222.0.88 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:42.829917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502d830-240b-47ca-96e2-e595e2e3fde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:42.835771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502d830-240b-47ca-96e2-e595e2e3fde2 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:42.835771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502d830-240b-47ca-96e2-e595e2e3fde2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:42.835771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502d830-240b-47ca-96e2-e595e2e3fde2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:42.835771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502d830-240b-47ca-96e2-e595e2e3fde2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:42.846658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 467/928] 10.222.0.88 () {56 vars in 1122 bytes} [Tue Aug 23 22:10:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:42.856784 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959c659-24f7-47c8-a066-0d4f40afa3af None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:42.962542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:43.069001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959c659-24f7-47c8-a066-0d4f40afa3af None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:43.136312 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68f03a-8a2a-4e71-9313-4dd0113264d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:43.137869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68f03a-8a2a-4e71-9313-4dd0113264d4 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:43.138660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68f03a-8a2a-4e71-9313-4dd0113264d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:43.139936 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68f03a-8a2a-4e71-9313-4dd0113264d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:43.141634 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68f03a-8a2a-4e71-9313-4dd0113264d4 None admin] RBAC: Adding query filter params (interface=public, service_id=290b82742f4043ce9b4eab314151e7cd, region_id=RegionOne) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:43.142492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68f03a-8a2a-4e71-9313-4dd0113264d4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:43.145453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68f03a-8a2a-4e71-9313-4dd0113264d4 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:43.156564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 462/929] 10.222.0.88 () {56 vars in 1251 bytes} [Tue Aug 23 22:10:42 2022] GET /identity/v3/endpoints?service_id=290b82742f4043ce9b4eab314151e7cd&interface=public®ion_id=RegionOne => generated 194 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:43.164586 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502d830-240b-47ca-96e2-e595e2e3fde2 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:43.175098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:43.176601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40880474-0933-4292-a173-bb71a0a8d396 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:43.177149 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40880474-0933-4292-a173-bb71a0a8d396 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:43.177651 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40880474-0933-4292-a173-bb71a0a8d396 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:43.185398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 463/930] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:43.190384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b14a76-a391-40b0-8c94-cd0256462e09 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:43.190863 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b14a76-a391-40b0-8c94-cd0256462e09 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:43.191341 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b14a76-a391-40b0-8c94-cd0256462e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:43.227275 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-88b14a76-a391-40b0-8c94-cd0256462e09 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:43.230048 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-88b14a76-a391-40b0-8c94-cd0256462e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:43.290713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502d830-240b-47ca-96e2-e595e2e3fde2 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:43.337740 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276b10e-49ce-4d94-83c9-5dff2dfd55bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:43.338457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276b10e-49ce-4d94-83c9-5dff2dfd55bf None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:43.338693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276b10e-49ce-4d94-83c9-5dff2dfd55bf None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:43.338961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276b10e-49ce-4d94-83c9-5dff2dfd55bf None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:43.339581 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276b10e-49ce-4d94-83c9-5dff2dfd55bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:43.341009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276b10e-49ce-4d94-83c9-5dff2dfd55bf None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:43.353625 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 468/931] 10.222.0.88 () {56 vars in 1087 bytes} [Tue Aug 23 22:10:43 2022] GET /identity/v3/services => generated 1497 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:10:43.355775 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2baea-b6f9-4603-8530-ce50cbd1bbd3 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:43.357572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2baea-b6f9-4603-8530-ce50cbd1bbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:43.358742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2baea-b6f9-4603-8530-ce50cbd1bbd3 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:43.361476 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 469/932] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:43.367674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03836d-bd5f-42a0-98a9-8e066d566852 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:43.368190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03836d-bd5f-42a0-98a9-8e066d566852 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:43.368683 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03836d-bd5f-42a0-98a9-8e066d566852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:43.391765 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 464/933] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:10:43.402449 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ab03836d-bd5f-42a0-98a9-8e066d566852 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:43.405304 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ab03836d-bd5f-42a0-98a9-8e066d566852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:43.488341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 470/934] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:43.776922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b14a76-a391-40b0-8c94-cd0256462e09 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:43.880343 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:43.946931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03836d-bd5f-42a0-98a9-8e066d566852 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:43.951189 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:43.985122 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b14a76-a391-40b0-8c94-cd0256462e09 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:44.049299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df6689-6c48-4942-8ec3-5d75ebed4ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:44.050278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df6689-6c48-4942-8ec3-5d75ebed4ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:44.050278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df6689-6c48-4942-8ec3-5d75ebed4ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:44.050573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df6689-6c48-4942-8ec3-5d75ebed4ad0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:44.063785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df6689-6c48-4942-8ec3-5d75ebed4ad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:44.068908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-c5df6689-6c48-4942-8ec3-5d75ebed4ad0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:10:44.068908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:44.068908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:44.068908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:44.068908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:44.068908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:44.068908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:44.068908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:44.068908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:44.068908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:44.068908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:44.068908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:44.068908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:44.068908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:44.068908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:44.068908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:44.070718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:44.070718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:44.070718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:44.070718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:44.070718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:44.070718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:44.070718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:44.070718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:44.070718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:44.070718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:44.070718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:44.070718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:44.070718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:44.070718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:44.070718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:44.070718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:44.072055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:44.072055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:44.072055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:44.072055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:44.072055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:44.072055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:44.072055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:44.072055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:44.072055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:44.072055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:44.072055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:10:44.072055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:44.072055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 465/935] 10.222.0.88 () {56 vars in 1120 bytes} [Tue Aug 23 22:10:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:44.077119 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df6689-6c48-4942-8ec3-5d75ebed4ad0 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:44.149764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03836d-bd5f-42a0-98a9-8e066d566852 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:44.191232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f72a3-cdde-49e2-a95a-8ce79cbc1ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:44.191909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f72a3-cdde-49e2-a95a-8ce79cbc1ddb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:44.192249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f72a3-cdde-49e2-a95a-8ce79cbc1ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:44.192525 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f72a3-cdde-49e2-a95a-8ce79cbc1ddb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:44.193142 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f72a3-cdde-49e2-a95a-8ce79cbc1ddb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:44.194596 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f72a3-cdde-49e2-a95a-8ce79cbc1ddb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:44.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-9c8f72a3-cdde-49e2-a95a-8ce79cbc1ddb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:10:44.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:44.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:44.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:44.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:44.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:44.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:44.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:44.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:10:44.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:44.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:44.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:44.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:44.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:44.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:44.205204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:44.206381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:44.207334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:44.207334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:44.207334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:44.207334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:44.207334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:44.207334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:44.207334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:44.207334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:44.207334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:44.207334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:44.207334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:44.207334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:44.207334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:10:44.207334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:44.207334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 471/936] 10.222.0.88 () {56 vars in 1132 bytes} [Tue Aug 23 22:10:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:44.216324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f72a3-cdde-49e2-a95a-8ce79cbc1ddb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:44.237616 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df6689-6c48-4942-8ec3-5d75ebed4ad0 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:44.251797 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:44.290128 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9b96e-fd23-43e3-a829-d62dd9b96d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:44.291207 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9b96e-fd23-43e3-a829-d62dd9b96d2b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:44.292385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9b96e-fd23-43e3-a829-d62dd9b96d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:44.292895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9b96e-fd23-43e3-a829-d62dd9b96d2b None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:44.293827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9b96e-fd23-43e3-a829-d62dd9b96d2b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:44.294370 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9b96e-fd23-43e3-a829-d62dd9b96d2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:44.296504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9b96e-fd23-43e3-a829-d62dd9b96d2b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:44.308879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 466/937] 10.222.0.88 () {56 vars in 1115 bytes} [Tue Aug 23 22:10:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:44.322915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9b96e-fd23-43e3-a829-d62dd9b96d2b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:44.345904 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:44.397935 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f72a3-cdde-49e2-a95a-8ce79cbc1ddb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:44.443456 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff38453-070d-4321-8f75-e876acaad769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:44.444445 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff38453-070d-4321-8f75-e876acaad769 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:44.444964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff38453-070d-4321-8f75-e876acaad769 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:44.445477 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff38453-070d-4321-8f75-e876acaad769 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:44.446397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff38453-070d-4321-8f75-e876acaad769 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:44.446949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff38453-070d-4321-8f75-e876acaad769 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:44.448648 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff38453-070d-4321-8f75-e876acaad769 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:44.455687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 472/938] 10.222.0.88 () {56 vars in 1126 bytes} [Tue Aug 23 22:10:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:44.465082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff38453-070d-4321-8f75-e876acaad769 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:44.504811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9b96e-fd23-43e3-a829-d62dd9b96d2b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:44.558551 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3c42f-f00d-42fa-9750-79577ad2a509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:44.560032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3c42f-f00d-42fa-9750-79577ad2a509 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:44.560544 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3c42f-f00d-42fa-9750-79577ad2a509 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:44.561896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3c42f-f00d-42fa-9750-79577ad2a509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:44.577818 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3c42f-f00d-42fa-9750-79577ad2a509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:44.580081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3c42f-f00d-42fa-9750-79577ad2a509 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:44.601827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-2bb3c42f-f00d-42fa-9750-79577ad2a509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:44.601827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:44.601827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:44.601827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:44.601827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:44.601827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:44.601827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:44.601827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:44.601827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:44.601827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:44.601827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:44.601827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:44.601827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:44.601827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:44.601827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:44.601827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:44.602879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:44.603830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:44.603830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:44.603830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:44.611497 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 467/939] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:44.614210 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3c42f-f00d-42fa-9750-79577ad2a509 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:44.648113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:44.694440 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff38453-070d-4321-8f75-e876acaad769 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:44.750415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:44.766295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a51da5-3223-4862-8b58-2a3a07aa0435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:44.767836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a51da5-3223-4862-8b58-2a3a07aa0435 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:44.768698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a51da5-3223-4862-8b58-2a3a07aa0435 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:44.769763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a51da5-3223-4862-8b58-2a3a07aa0435 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:44.771432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a51da5-3223-4862-8b58-2a3a07aa0435 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:44.772328 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a51da5-3223-4862-8b58-2a3a07aa0435 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:44.775827 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a51da5-3223-4862-8b58-2a3a07aa0435 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:44.786723 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 473/940] 10.222.0.88 () {56 vars in 1126 bytes} [Tue Aug 23 22:10:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:44.796480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a51da5-3223-4862-8b58-2a3a07aa0435 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:44.815200 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3c42f-f00d-42fa-9750-79577ad2a509 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:44.883177 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da84e4-248f-4e43-9ae1-c5a7f45acbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:44.884269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da84e4-248f-4e43-9ae1-c5a7f45acbca None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:44.884857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da84e4-248f-4e43-9ae1-c5a7f45acbca None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:44.885446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da84e4-248f-4e43-9ae1-c5a7f45acbca None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:44.886806 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da84e4-248f-4e43-9ae1-c5a7f45acbca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:44.888006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da84e4-248f-4e43-9ae1-c5a7f45acbca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:44.890465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da84e4-248f-4e43-9ae1-c5a7f45acbca None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:44.925608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 468/941] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:44.928724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da84e4-248f-4e43-9ae1-c5a7f45acbca None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:44.943616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a51da5-3223-4862-8b58-2a3a07aa0435 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:44.987101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebff10-ca42-4232-a970-12e749f4d9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:44.988414 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebff10-ca42-4232-a970-12e749f4d9fd None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:44.989124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebff10-ca42-4232-a970-12e749f4d9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:44.989623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebff10-ca42-4232-a970-12e749f4d9fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:44.990934 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ebff10-ca42-4232-a970-12e749f4d9fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:44.993456 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ebff10-ca42-4232-a970-12e749f4d9fd None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:45.030534 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 474/942] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:10:44 2022] POST /identity/v3/endpoints => generated 347 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:10:45.088940 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2851b70-edf1-4fb3-b900-8d8ad30f7d99 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:45.089383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2851b70-edf1-4fb3-b900-8d8ad30f7d99 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:45.089791 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2851b70-edf1-4fb3-b900-8d8ad30f7d99 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:45.091568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 475/943] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:45.099005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a6bd5-26e0-46ec-9abc-023314461701 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:45.099285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a6bd5-26e0-46ec-9abc-023314461701 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:45.099629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a6bd5-26e0-46ec-9abc-023314461701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:45.117900 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:45.128280 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-bc9a6bd5-26e0-46ec-9abc-023314461701 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:45.130276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bc9a6bd5-26e0-46ec-9abc-023314461701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:45.148961 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da84e4-248f-4e43-9ae1-c5a7f45acbca None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:45.227885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:45.231506 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0e5c5-77c8-49fd-b6bd-495a94de7c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:45.233703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e5c5-77c8-49fd-b6bd-495a94de7c99 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:45.234415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e5c5-77c8-49fd-b6bd-495a94de7c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:45.235101 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e5c5-77c8-49fd-b6bd-495a94de7c99 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:45.248588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0e5c5-77c8-49fd-b6bd-495a94de7c99 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:45.251387 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0e5c5-77c8-49fd-b6bd-495a94de7c99 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:45.267022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-b9a0e5c5-77c8-49fd-b6bd-495a94de7c99 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:10:45.267022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:45.267022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:45.267022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:45.267022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:45.267022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:45.267022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:45.267022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:45.267022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:45.267022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:45.267022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:45.267022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:45.267022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:45.267022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:45.267022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:45.267022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:45.268688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:45.270579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:45.270579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:45.270579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:45.270579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:45.270579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:45.270579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:45.270579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:45.270579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:45.270579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:10:45.270579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:45.270579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 469/944] 10.222.0.88 () {56 vars in 1117 bytes} [Tue Aug 23 22:10:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:45.273990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 476/945] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:10:45.283110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0e5c5-77c8-49fd-b6bd-495a94de7c99 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:45.411342 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0e5c5-77c8-49fd-b6bd-495a94de7c99 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:45.455947 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9095f5-e633-4dae-a15e-29571ab64628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:45.456852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9095f5-e633-4dae-a15e-29571ab64628 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:45.457306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9095f5-e633-4dae-a15e-29571ab64628 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:45.457767 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9095f5-e633-4dae-a15e-29571ab64628 None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:45.458703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9095f5-e633-4dae-a15e-29571ab64628 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:45.459270 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9095f5-e633-4dae-a15e-29571ab64628 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:45.460969 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9095f5-e633-4dae-a15e-29571ab64628 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:45.472964 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:45.485553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 470/946] 10.222.0.88 () {56 vars in 1149 bytes} [Tue Aug 23 22:10:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:45.500882 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a6bd5-26e0-46ec-9abc-023314461701 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:45.727486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a6bd5-26e0-46ec-9abc-023314461701 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:45.792966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f937083-662b-4280-9833-89098f6d3253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:45.794388 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f937083-662b-4280-9833-89098f6d3253 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:45.795032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f937083-662b-4280-9833-89098f6d3253 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:45.795636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f937083-662b-4280-9833-89098f6d3253 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:45.811927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9095f5-e633-4dae-a15e-29571ab64628 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:45.820420 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f937083-662b-4280-9833-89098f6d3253 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:45.823358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f937083-662b-4280-9833-89098f6d3253 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:45.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-0f937083-662b-4280-9833-89098f6d3253 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:45.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:45.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:45.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:45.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:45.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:45.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:45.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:45.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:45.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:45.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:45.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:45.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:45.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:45.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:45.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:45.854765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:45.856792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:45.856792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:45.856792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:45.865106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 477/947] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:45.868138 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f937083-662b-4280-9833-89098f6d3253 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:45.928882 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:46.004710 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9095f5-e633-4dae-a15e-29571ab64628 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:46.067644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f937083-662b-4280-9833-89098f6d3253 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:46.073035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66623d73-03eb-4d41-84f6-85a084678bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:46.074050 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66623d73-03eb-4d41-84f6-85a084678bf8 None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:46.074495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66623d73-03eb-4d41-84f6-85a084678bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:46.074934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66623d73-03eb-4d41-84f6-85a084678bf8 None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:46.076003 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66623d73-03eb-4d41-84f6-85a084678bf8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:46.078808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66623d73-03eb-4d41-84f6-85a084678bf8 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:46.105470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.sql.core [None req-66623d73-03eb-4d41-84f6-85a084678bf8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 22:10:46.105470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 23 22:10:46.105470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [parameters: {'user_id': 'd33e8ddb56a34b9aacf237d769e0e181', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 23 22:10:46.105470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81679) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 22:10:46.121104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-66623d73-03eb-4d41-84f6-85a084678bf8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 23 22:10:46.121104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:46.121104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 22:10:46.121104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 22:10:46.121104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 22:10:46.121104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 22:10:46.121104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 22:10:46.121104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 22:10:46.121104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 22:10:46.121104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application conn.query(q) Aug 23 22:10:46.121104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 22:10:46.121104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 22:10:46.121104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 22:10:46.121104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application result.read() Aug 23 22:10:46.121104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 22:10:46.122110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application next(self.gen) Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application yield resource Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application next(self.gen) Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise self.value Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application session.commit() Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 22:10:46.123206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self.session.flush() Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application compat.raise_( Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 22:10:46.124313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 22:10:46.125327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 22:10:46.125327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 22:10:46.125327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 23 22:10:46.125327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 23 22:10:46.125327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 22:10:46.125327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 22:10:46.125327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 22:10:46.125327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 22:10:46.125327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 22:10:46.125327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 22:10:46.125327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 22:10:46.125327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 22:10:46.125327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 22:10:46.125327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 22:10:46.125327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 22:10:46.125327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception Aug 23 22:10:46.126396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 22:10:46.126396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 22:10:46.126396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 22:10:46.126396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 22:10:46.126396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 22:10:46.126396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 22:10:46.126396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 22:10:46.126396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application conn.query(q) Aug 23 22:10:46.126396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 22:10:46.126396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 22:10:46.126396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 22:10:46.126396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application result.read() Aug 23 22:10:46.126396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 22:10:46.126396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 22:10:46.126396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 22:10:46.126396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 22:10:46.128349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 22:10:46.128349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 22:10:46.128349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 22:10:46.128349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 22:10:46.128349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 22:10:46.128349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 23 22:10:46.128349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd33e8ddb56a34b9aacf237d769e0e181', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 23 22:10:46.128349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 22:10:46.128349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:46.128349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:46.128349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:46.128349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:46.128349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:46.128349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:46.128349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 23 22:10:46.130548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 23 22:10:46.132902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 23 22:10:46.132902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 23 22:10:46.132902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 22:10:46.132902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 22:10:46.132902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 23 22:10:46.132902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:46.132902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 471/948] 10.222.0.88 () {60 vars in 1131 bytes} [Tue Aug 23 22:10:45 2022] POST /identity/v3/users => generated 160 bytes in 310 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:46.132902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c26f5-c7bd-4863-a4ac-0a53265e8d9e None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:46.132902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c26f5-c7bd-4863-a4ac-0a53265e8d9e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:46.132902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c26f5-c7bd-4863-a4ac-0a53265e8d9e None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:46.134471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dfaba-6d4a-4bee-8e50-381cb3b06bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:46.134471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dfaba-6d4a-4bee-8e50-381cb3b06bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:46.134471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dfaba-6d4a-4bee-8e50-381cb3b06bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:46.134935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 472/949] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:46 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:46.135141 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dfaba-6d4a-4bee-8e50-381cb3b06bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:46.135141 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220dfaba-6d4a-4bee-8e50-381cb3b06bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:46.135416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c26f5-c7bd-4863-a4ac-0a53265e8d9e None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:46.135591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220dfaba-6d4a-4bee-8e50-381cb3b06bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:46.137986 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220dfaba-6d4a-4bee-8e50-381cb3b06bf3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:46.159708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 478/950] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:46.161891 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c4f4b-afe8-4817-b6c8-d3b848baea10 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:46.162376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c4f4b-afe8-4817-b6c8-d3b848baea10 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:46.162847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c4f4b-afe8-4817-b6c8-d3b848baea10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:46.187107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-769c4f4b-afe8-4817-b6c8-d3b848baea10 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:46.188836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-769c4f4b-afe8-4817-b6c8-d3b848baea10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:46.223670 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:46.275525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c26f5-c7bd-4863-a4ac-0a53265e8d9e None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:46.282093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 479/951] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:46.283146 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c4f4b-afe8-4817-b6c8-d3b848baea10 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:46.322443 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:46.347115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4f1a6-d5c6-494a-be8e-b4d828ffb826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:46.348523 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4f1a6-d5c6-494a-be8e-b4d828ffb826 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:46.349495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4f1a6-d5c6-494a-be8e-b4d828ffb826 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:46.350182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4f1a6-d5c6-494a-be8e-b4d828ffb826 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:46.368080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4f1a6-d5c6-494a-be8e-b4d828ffb826 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:46.371752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4f1a6-d5c6-494a-be8e-b4d828ffb826 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:46.398721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-ffa4f1a6-d5c6-494a-be8e-b4d828ffb826 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 22:10:46.398721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:46.398721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:46.398721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:46.398721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:46.398721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:46.398721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:46.398721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:46.398721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:46.398721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:46.398721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:46.398721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:46.398721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:46.398721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:46.398721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:46.398721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:46.400758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:46.403091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:46.403091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:46.403091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:46.403091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:46.403091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:46.403091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:46.403091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:46.403091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:46.403091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 22:10:46.403091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:46.413742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 473/952] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:46.417654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4f1a6-d5c6-494a-be8e-b4d828ffb826 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:46.463978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c4f4b-afe8-4817-b6c8-d3b848baea10 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:46.521848 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad34eb-79bf-40c5-ab9e-d191b08245cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:46.523835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad34eb-79bf-40c5-ab9e-d191b08245cd None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:46.524291 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad34eb-79bf-40c5-ab9e-d191b08245cd None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:46.524713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad34eb-79bf-40c5-ab9e-d191b08245cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:46.536338 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad34eb-79bf-40c5-ab9e-d191b08245cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:46.537780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad34eb-79bf-40c5-ab9e-d191b08245cd None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:46.552777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-3dad34eb-79bf-40c5-ab9e-d191b08245cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:46.552777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:46.552777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:46.552777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:46.552777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:46.552777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:46.552777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:46.552777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:46.552777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:46.552777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:46.552777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:46.552777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:46.552777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:46.552777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:46.552777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:46.554764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:46.556917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:46.556917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:46.556917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:46.556917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:46.556917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:46.556917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:46.556917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:46.556917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:46.559356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 480/953] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:46.562147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad34eb-79bf-40c5-ab9e-d191b08245cd None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:46.652784 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4f1a6-d5c6-494a-be8e-b4d828ffb826 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:46.678176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:46.698544 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:46.702002 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7ab60-4dcb-4079-bd54-ba9e3d113025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:46.702865 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7ab60-4dcb-4079-bd54-ba9e3d113025 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:46.703585 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7ab60-4dcb-4079-bd54-ba9e3d113025 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:46.704007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7ab60-4dcb-4079-bd54-ba9e3d113025 None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:46.705820 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7ab60-4dcb-4079-bd54-ba9e3d113025 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:46.706365 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7ab60-4dcb-4079-bd54-ba9e3d113025 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:46.708113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7ab60-4dcb-4079-bd54-ba9e3d113025 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:46.734510 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad34eb-79bf-40c5-ab9e-d191b08245cd None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:46.761123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 474/954] 10.222.0.88 () {56 vars in 1101 bytes} [Tue Aug 23 22:10:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:46.761123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7ab60-4dcb-4079-bd54-ba9e3d113025 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:46.821874 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81c685-9280-4edc-bb98-3c71235b640d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:46.823118 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81c685-9280-4edc-bb98-3c71235b640d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:46.823701 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81c685-9280-4edc-bb98-3c71235b640d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:46.824276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81c685-9280-4edc-bb98-3c71235b640d None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:46.825923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81c685-9280-4edc-bb98-3c71235b640d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:46.827161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81c685-9280-4edc-bb98-3c71235b640d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:46.830004 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81c685-9280-4edc-bb98-3c71235b640d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:46.861749 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 481/955] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:46.864616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81c685-9280-4edc-bb98-3c71235b640d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:46.992771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7ab60-4dcb-4079-bd54-ba9e3d113025 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:47.045938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81c685-9280-4edc-bb98-3c71235b640d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:47.057021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23386ca2-d38c-40a6-a7f3-d19248739b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:47.057843 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23386ca2-d38c-40a6-a7f3-d19248739b8d None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.058821 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23386ca2-d38c-40a6-a7f3-d19248739b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.058821 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23386ca2-d38c-40a6-a7f3-d19248739b8d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.059402 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23386ca2-d38c-40a6-a7f3-d19248739b8d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:47.062166 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23386ca2-d38c-40a6-a7f3-d19248739b8d None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-23386ca2-d38c-40a6-a7f3-d19248739b8d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:47.071904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:47.074056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:47.074056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:47.074056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:47.074056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:47.074056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:47.074056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:47.074056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:47.074056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:47.074056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:47.074056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:47.074056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:47.074056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:47.074056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:47.074056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:47.074056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:47.074056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:47.075724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:47.075724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:47.075724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:47.075724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:47.075724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:47.075724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:47.075724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:47.075724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:47.075724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:47.075724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:47.075724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:47.075724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:47.075724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:10:47.075724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:47.075724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 475/956] 10.222.0.88 () {56 vars in 1126 bytes} [Tue Aug 23 22:10:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:47.081449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23386ca2-d38c-40a6-a7f3-d19248739b8d None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:47.111125 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8d2d0-26bf-456f-967d-4439708d284e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:47.111799 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8d2d0-26bf-456f-967d-4439708d284e None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.112096 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8d2d0-26bf-456f-967d-4439708d284e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.112354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8d2d0-26bf-456f-967d-4439708d284e None admin] PATH_INFO: `/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/cb001b14eec54047aee42ca6bd4a1eba/roles/97c55e5f00404a29bb3f2df9566b6b0f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.114571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8d2d0-26bf-456f-967d-4439708d284e None admin] RBAC: Authorizing `identity:create_grant(project_id=8a7317af325f485e8fae49631cb5bc20, user_id=cb001b14eec54047aee42ca6bd4a1eba, role_id=97c55e5f00404a29bb3f2df9566b6b0f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:47.116568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8d2d0-26bf-456f-967d-4439708d284e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:47.144391 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:47.149114 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 482/957] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:10:46 2022] PUT /identity/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/cb001b14eec54047aee42ca6bd4a1eba/roles/97c55e5f00404a29bb3f2df9566b6b0f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:10:47.228589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23386ca2-d38c-40a6-a7f3-d19248739b8d None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:47.294279 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41774a8b-5633-44de-bfcb-8bd32182fb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:47.296127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41774a8b-5633-44de-bfcb-8bd32182fb55 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.296127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41774a8b-5633-44de-bfcb-8bd32182fb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.296643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41774a8b-5633-44de-bfcb-8bd32182fb55 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.297999 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41774a8b-5633-44de-bfcb-8bd32182fb55 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:47.299361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41774a8b-5633-44de-bfcb-8bd32182fb55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:47.301218 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41774a8b-5633-44de-bfcb-8bd32182fb55 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:47.311429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 476/958] 10.222.0.88 () {56 vars in 1122 bytes} [Tue Aug 23 22:10:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:47.318401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8d2d0-26bf-456f-967d-4439708d284e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:47.322032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:47.391118 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce1935-afd5-4adc-8aa0-d94bf1ba168a None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.391382 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce1935-afd5-4adc-8aa0-d94bf1ba168a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.391661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce1935-afd5-4adc-8aa0-d94bf1ba168a None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.393185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 477/959] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:47.399349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d53def-24ee-4242-b9ab-38e5d1e19a64 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.399931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d53def-24ee-4242-b9ab-38e5d1e19a64 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.400334 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d53def-24ee-4242-b9ab-38e5d1e19a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.422247 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-51d53def-24ee-4242-b9ab-38e5d1e19a64 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:47.423918 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-51d53def-24ee-4242-b9ab-38e5d1e19a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:47.471100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8d2d0-26bf-456f-967d-4439708d284e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:47.524444 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595bdf2-1157-4de3-8538-9255d68c80ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:47.525470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595bdf2-1157-4de3-8538-9255d68c80ca None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.525916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595bdf2-1157-4de3-8538-9255d68c80ca None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.526393 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595bdf2-1157-4de3-8538-9255d68c80ca None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.527149 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595bdf2-1157-4de3-8538-9255d68c80ca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:47.527890 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595bdf2-1157-4de3-8538-9255d68c80ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:47.530743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595bdf2-1157-4de3-8538-9255d68c80ca None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:47.533111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 478/960] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:10:47.548440 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 483/961] 10.222.0.88 () {56 vars in 1122 bytes} [Tue Aug 23 22:10:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:47.550178 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d53def-24ee-4242-b9ab-38e5d1e19a64 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:47.594136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:47.668993 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d53def-24ee-4242-b9ab-38e5d1e19a64 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:47.708061 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6f904-2e68-40ec-857f-12214e7fe8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:47.708841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6f904-2e68-40ec-857f-12214e7fe8d9 None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.709251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6f904-2e68-40ec-857f-12214e7fe8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.709621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6f904-2e68-40ec-857f-12214e7fe8d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.710403 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6f904-2e68-40ec-857f-12214e7fe8d9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:47.711882 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6f904-2e68-40ec-857f-12214e7fe8d9 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:47.735081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 479/962] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:10:47 2022] POST /identity/v3/endpoints => generated 349 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:47.991542 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595bdf2-1157-4de3-8538-9255d68c80ca None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:48.086694 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:48.159815 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595bdf2-1157-4de3-8538-9255d68c80ca None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:48.236041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e44bb8-4269-42b4-8590-9c32714a0e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:48.237030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e44bb8-4269-42b4-8590-9c32714a0e69 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:48.237501 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e44bb8-4269-42b4-8590-9c32714a0e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:48.237919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e44bb8-4269-42b4-8590-9c32714a0e69 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:48.238653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e44bb8-4269-42b4-8590-9c32714a0e69 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:48.240211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e44bb8-4269-42b4-8590-9c32714a0e69 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-03e44bb8-4269-42b4-8590-9c32714a0e69 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:48.250601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:48.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:48.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:48.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:48.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:48.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:48.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:48.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:48.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:48.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:48.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:48.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:48.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:48.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:48.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:48.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:48.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:48.253650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:48.253650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:48.253650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:48.253650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:48.253650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:48.253650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:48.253650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:48.253650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:48.253650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:48.253650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:48.253650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:48.253650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:48.253650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:10:48.253650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:48.257354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 484/963] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:48.262738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6f904-2e68-40ec-857f-12214e7fe8d9 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:48.423409 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:48.441985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6f904-2e68-40ec-857f-12214e7fe8d9 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:48.503414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcdf8d1-342c-4722-b081-d2c2dbaa036e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:48.504367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdf8d1-342c-4722-b081-d2c2dbaa036e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:48.504844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdf8d1-342c-4722-b081-d2c2dbaa036e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:48.505368 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdf8d1-342c-4722-b081-d2c2dbaa036e None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:48.506500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcdf8d1-342c-4722-b081-d2c2dbaa036e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:48.507038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcdf8d1-342c-4722-b081-d2c2dbaa036e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:48.509560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcdf8d1-342c-4722-b081-d2c2dbaa036e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:48.524203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 480/964] 10.222.0.88 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:48.526173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e44bb8-4269-42b4-8590-9c32714a0e69 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:48.676154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e44bb8-4269-42b4-8590-9c32714a0e69 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:48.717716 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:48.735818 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648639e4-3be5-489e-8f43-21b476b321e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:48.737085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648639e4-3be5-489e-8f43-21b476b321e1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:48.737703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648639e4-3be5-489e-8f43-21b476b321e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:48.738317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648639e4-3be5-489e-8f43-21b476b321e1 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:48.739672 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648639e4-3be5-489e-8f43-21b476b321e1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:48.740366 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648639e4-3be5-489e-8f43-21b476b321e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:48.743208 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648639e4-3be5-489e-8f43-21b476b321e1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:48.752599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 485/965] 10.222.0.88 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:48 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:49.299331 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b7643-320f-4503-8749-1e22fe95ff7b None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:49.300142 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b7643-320f-4503-8749-1e22fe95ff7b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:49.300142 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b7643-320f-4503-8749-1e22fe95ff7b None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:49.301453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 481/966] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:49.307109 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a4d51-6d70-4957-a869-bc7d241c6e37 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:49.307431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a4d51-6d70-4957-a869-bc7d241c6e37 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:49.307663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a4d51-6d70-4957-a869-bc7d241c6e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:49.328401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9d4a4d51-6d70-4957-a869-bc7d241c6e37 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:49.329662 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9d4a4d51-6d70-4957-a869-bc7d241c6e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:49.413863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 486/967] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:49.703701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b7643-320f-4503-8749-1e22fe95ff7b None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:49.865095 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b7643-320f-4503-8749-1e22fe95ff7b None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:49.923312 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9cacf-9ccb-45bd-b691-129f2a643f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:49.924258 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9cacf-9ccb-45bd-b691-129f2a643f53 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:49.924702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9cacf-9ccb-45bd-b691-129f2a643f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:49.925173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9cacf-9ccb-45bd-b691-129f2a643f53 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:49.945366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a9cacf-9ccb-45bd-b691-129f2a643f53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:49.954228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-f8a9cacf-9ccb-45bd-b691-129f2a643f53 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:10:49.954228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:49.954228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:49.954228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:49.954228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:49.954228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:49.954228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:49.954228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:49.954228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:49.954228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:49.954228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:49.954228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:49.954228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:49.954228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:49.954228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:49.954228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:49.955306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:49.955306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:49.955306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:49.955306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:49.955306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:49.955306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:49.955306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:49.955306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:49.955306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:49.955306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:49.955306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:49.955306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:49.955306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:49.955306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:49.955306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:49.955306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:49.961392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:49.961392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:49.961392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:49.961392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:49.961392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:49.961392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:49.961392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:49.961392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:49.961392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:49.961392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:49.961392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:10:49.961392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:49.961392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 482/968] 10.222.0.88 () {56 vars in 1120 bytes} [Tue Aug 23 22:10:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:49.963068 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a4d51-6d70-4957-a869-bc7d241c6e37 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:50.132758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a4d51-6d70-4957-a869-bc7d241c6e37 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:50.170334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:50.195035 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb09e9c-93f6-4481-9609-b89183d8db7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:50.195982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb09e9c-93f6-4481-9609-b89183d8db7d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:50.196465 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb09e9c-93f6-4481-9609-b89183d8db7d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:50.196910 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb09e9c-93f6-4481-9609-b89183d8db7d None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:50.198155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb09e9c-93f6-4481-9609-b89183d8db7d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:50.198674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb09e9c-93f6-4481-9609-b89183d8db7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:50.201636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb09e9c-93f6-4481-9609-b89183d8db7d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:50.221556 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 487/969] 10.222.0.88 () {56 vars in 1115 bytes} [Tue Aug 23 22:10:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:50.223508 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9cacf-9ccb-45bd-b691-129f2a643f53 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:50.250220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:50.401789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9cacf-9ccb-45bd-b691-129f2a643f53 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:50.466556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1c91b-4afb-4187-bb2b-5d9db54df11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:50.468237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1c91b-4afb-4187-bb2b-5d9db54df11e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:50.468712 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1c91b-4afb-4187-bb2b-5d9db54df11e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:50.469214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1c91b-4afb-4187-bb2b-5d9db54df11e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:50.491976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1c91b-4afb-4187-bb2b-5d9db54df11e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:50.494577 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1c91b-4afb-4187-bb2b-5d9db54df11e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:50.523520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-84b1c91b-4afb-4187-bb2b-5d9db54df11e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:50.523520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:50.523520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:50.523520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:50.523520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:50.523520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:50.523520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:50.523520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:50.523520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:50.523520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:50.523520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:50.523520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:50.523520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:50.523520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:50.523520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:50.523520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:50.524572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:50.526124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:50.526124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:50.526124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:50.526124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 483/970] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:50.533370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb09e9c-93f6-4481-9609-b89183d8db7d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:50.662525 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb09e9c-93f6-4481-9609-b89183d8db7d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:50.703108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a078e0e-f653-4c7d-967d-2eb279dc8797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:50.703717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a078e0e-f653-4c7d-967d-2eb279dc8797 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:50.704010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a078e0e-f653-4c7d-967d-2eb279dc8797 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:50.704284 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a078e0e-f653-4c7d-967d-2eb279dc8797 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:50.704909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a078e0e-f653-4c7d-967d-2eb279dc8797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:50.705275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a078e0e-f653-4c7d-967d-2eb279dc8797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:50.706705 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a078e0e-f653-4c7d-967d-2eb279dc8797 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:50.711019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1825e1-8340-4ebf-8f0e-5688b280105a None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:50.711493 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1825e1-8340-4ebf-8f0e-5688b280105a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:50.711981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1825e1-8340-4ebf-8f0e-5688b280105a None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:50.714771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 484/971] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:50.729109 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 488/972] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:50.729406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6919610-9e79-4366-88c5-b86304f4d282 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:50.729406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6919610-9e79-4366-88c5-b86304f4d282 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:50.729406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6919610-9e79-4366-88c5-b86304f4d282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:50.731844 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a078e0e-f653-4c7d-967d-2eb279dc8797 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:50.757038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-f6919610-9e79-4366-88c5-b86304f4d282 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:50.759069 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f6919610-9e79-4366-88c5-b86304f4d282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:50.806712 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:50.870328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:50.882942 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 485/973] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:10:50.922242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a078e0e-f653-4c7d-967d-2eb279dc8797 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:50.989744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6ff81-69fa-4865-a29f-801022fa05f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:50.990677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6ff81-69fa-4865-a29f-801022fa05f6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:50.991123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6ff81-69fa-4865-a29f-801022fa05f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:50.991560 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6ff81-69fa-4865-a29f-801022fa05f6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.006451 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6ff81-69fa-4865-a29f-801022fa05f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:51.009281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6ff81-69fa-4865-a29f-801022fa05f6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:51.034049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-59c6ff81-69fa-4865-a29f-801022fa05f6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:10:51.034049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:51.034049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:51.034049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:51.034049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:51.034049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:51.034049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:51.034049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:51.034049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:51.034049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:51.034049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:51.034049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:51.034049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:51.034049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:51.034049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:51.034049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:51.035989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:51.037744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:51.037744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:51.037744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:51.037744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:51.037744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:51.037744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:51.037744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:51.037744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:51.037744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:10:51.037744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:51.041660 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 489/974] 10.222.0.88 () {56 vars in 1117 bytes} [Tue Aug 23 22:10:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:51.046974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6919610-9e79-4366-88c5-b86304f4d282 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:51.055907 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2dacb-705c-46fa-9ab2-aa0543ec2e00 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.056274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2dacb-705c-46fa-9ab2-aa0543ec2e00 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.056631 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2dacb-705c-46fa-9ab2-aa0543ec2e00 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.058210 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 490/975] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:51.065858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28034984-e43c-43e4-9dba-d62fc92391a1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.066297 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28034984-e43c-43e4-9dba-d62fc92391a1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.066619 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28034984-e43c-43e4-9dba-d62fc92391a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.096859 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-28034984-e43c-43e4-9dba-d62fc92391a1 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:51.098768 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-28034984-e43c-43e4-9dba-d62fc92391a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:51.221369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 491/976] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:10:51.222471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28034984-e43c-43e4-9dba-d62fc92391a1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:51.230953 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6919610-9e79-4366-88c5-b86304f4d282 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:51.282223 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1edc8-5fba-4e94-9083-3ef0d4639ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:51.283100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1edc8-5fba-4e94-9083-3ef0d4639ddd None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.283560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1edc8-5fba-4e94-9083-3ef0d4639ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.283953 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1edc8-5fba-4e94-9083-3ef0d4639ddd None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.285217 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1edc8-5fba-4e94-9083-3ef0d4639ddd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:51.285728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1edc8-5fba-4e94-9083-3ef0d4639ddd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:51.287885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1edc8-5fba-4e94-9083-3ef0d4639ddd None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:51.288309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:51.321166 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 486/977] 10.222.0.88 () {56 vars in 1149 bytes} [Tue Aug 23 22:10:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:51.331604 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1edc8-5fba-4e94-9083-3ef0d4639ddd None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:51.403765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28034984-e43c-43e4-9dba-d62fc92391a1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:51.435138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:51.470400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ede25c-a465-4a3b-a737-f55fc3a16187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:51.471414 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ede25c-a465-4a3b-a737-f55fc3a16187 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.472189 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ede25c-a465-4a3b-a737-f55fc3a16187 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.472189 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ede25c-a465-4a3b-a737-f55fc3a16187 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.472746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ede25c-a465-4a3b-a737-f55fc3a16187 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:51.474922 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ede25c-a465-4a3b-a737-f55fc3a16187 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-94ede25c-a465-4a3b-a737-f55fc3a16187 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:51.490412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:51.491734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:51.491734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:51.491734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:51.491734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:51.491734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:51.491734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:51.491734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:51.491734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:51.491734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:51.491734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:51.491734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:51.491734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:51.491734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:51.491734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:51.491734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:51.491734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:51.493085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:51.493085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:51.493085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:51.493085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:51.493085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:51.493085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:51.493085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:51.493085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:51.493085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:51.493085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:51.493085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:51.493085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:51.493085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:10:51.493085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:51.493085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 492/978] 10.222.0.88 () {56 vars in 1126 bytes} [Tue Aug 23 22:10:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:51.499773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ede25c-a465-4a3b-a737-f55fc3a16187 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:51.509719 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1edc8-5fba-4e94-9083-3ef0d4639ddd None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:51.564934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf92ba4-ca65-45d2-82ca-9bca76f93bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:51.565580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf92ba4-ca65-45d2-82ca-9bca76f93bbf None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.565924 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf92ba4-ca65-45d2-82ca-9bca76f93bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.566280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf92ba4-ca65-45d2-82ca-9bca76f93bbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.648032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:51.656270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ede25c-a465-4a3b-a737-f55fc3a16187 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:51.684130 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf92ba4-ca65-45d2-82ca-9bca76f93bbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:51.686141 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf92ba4-ca65-45d2-82ca-9bca76f93bbf None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:51.695636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1da1eb-d217-4444-8d09-696ed671d5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:51.696263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1da1eb-d217-4444-8d09-696ed671d5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.696572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1da1eb-d217-4444-8d09-696ed671d5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.696852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1da1eb-d217-4444-8d09-696ed671d5a5 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.697543 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1da1eb-d217-4444-8d09-696ed671d5a5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:51.697850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1da1eb-d217-4444-8d09-696ed671d5a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:51.699309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1da1eb-d217-4444-8d09-696ed671d5a5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:51.707667 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 493/979] 10.222.0.88 () {56 vars in 1122 bytes} [Tue Aug 23 22:10:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:51.708877 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1da1eb-d217-4444-8d09-696ed671d5a5 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:51.759296 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:51.830948 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 487/980] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:51 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 22:10:51.831943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf92ba4-ca65-45d2-82ca-9bca76f93bbf None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:51.869337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1da1eb-d217-4444-8d09-696ed671d5a5 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:51.916313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2be0a-0ad6-435f-ab1d-783a18e8e898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:51.917210 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2be0a-0ad6-435f-ab1d-783a18e8e898 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.917550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2be0a-0ad6-435f-ab1d-783a18e8e898 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.917892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2be0a-0ad6-435f-ab1d-783a18e8e898 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.918744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd2be0a-0ad6-435f-ab1d-783a18e8e898 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:51.921043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd2be0a-0ad6-435f-ab1d-783a18e8e898 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:51.943325 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 494/981] 10.222.0.88 () {60 vars in 1141 bytes} [Tue Aug 23 22:10:51 2022] POST /identity/v3/services => generated 242 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:51.944556 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2be0a-0ad6-435f-ab1d-783a18e8e898 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:51.978487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:52.002219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf92ba4-ca65-45d2-82ca-9bca76f93bbf None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:52.054836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe985bc0-47c7-46b3-8ec9-f0b5e005a672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:52.055498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe985bc0-47c7-46b3-8ec9-f0b5e005a672 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:52.055945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe985bc0-47c7-46b3-8ec9-f0b5e005a672 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:52.056097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe985bc0-47c7-46b3-8ec9-f0b5e005a672 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:52.056830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe985bc0-47c7-46b3-8ec9-f0b5e005a672 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:52.057173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe985bc0-47c7-46b3-8ec9-f0b5e005a672 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:52.058763 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe985bc0-47c7-46b3-8ec9-f0b5e005a672 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:52.067146 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 488/982] 10.222.0.88 () {56 vars in 1122 bytes} [Tue Aug 23 22:10:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:52.073565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe985bc0-47c7-46b3-8ec9-f0b5e005a672 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:52.093780 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:52.130460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2be0a-0ad6-435f-ab1d-783a18e8e898 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:52.186913 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a994c-61a5-469a-b831-1df00421ecb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:52.187614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a994c-61a5-469a-b831-1df00421ecb6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:52.187892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a994c-61a5-469a-b831-1df00421ecb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:52.188262 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a994c-61a5-469a-b831-1df00421ecb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:52.200712 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a994c-61a5-469a-b831-1df00421ecb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:52.202263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a994c-61a5-469a-b831-1df00421ecb6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:52.219043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-da4a994c-61a5-469a-b831-1df00421ecb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:52.219043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:52.219043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:52.219043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:52.219043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:52.219043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:52.219043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:52.219043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:52.219043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:52.219043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:52.219043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:52.219043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:52.219043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:52.219043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:52.219043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:52.219043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:52.220147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:52.221285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:52.221285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:52.221285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:52.221285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 495/983] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:52.226067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a994c-61a5-469a-b831-1df00421ecb6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:52.231714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe985bc0-47c7-46b3-8ec9-f0b5e005a672 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:52.288087 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b93e7-e296-43ae-a02e-39647af611f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:52.289140 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b93e7-e296-43ae-a02e-39647af611f7 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:52.289629 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b93e7-e296-43ae-a02e-39647af611f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:52.290123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b93e7-e296-43ae-a02e-39647af611f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:52.291375 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b93e7-e296-43ae-a02e-39647af611f7 None admin] RBAC: Adding query filter params (interface=admin, service_id=290b82742f4043ce9b4eab314151e7cd, region_id=RegionOne) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:52.291928 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b93e7-e296-43ae-a02e-39647af611f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:52.294488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b93e7-e296-43ae-a02e-39647af611f7 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:52.306687 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 489/984] 10.222.0.88 () {56 vars in 1249 bytes} [Tue Aug 23 22:10:52 2022] GET /identity/v3/endpoints?service_id=290b82742f4043ce9b4eab314151e7cd&interface=admin®ion_id=RegionOne => generated 193 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:52.316119 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b93e7-e296-43ae-a02e-39647af611f7 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:52.322290 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:52.379299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a994c-61a5-469a-b831-1df00421ecb6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:52.421296 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:52.423595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be48837-4b7f-440f-9d31-a9eec88935f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:52.424561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be48837-4b7f-440f-9d31-a9eec88935f9 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:52.425177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be48837-4b7f-440f-9d31-a9eec88935f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:52.425655 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be48837-4b7f-440f-9d31-a9eec88935f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:52.426938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be48837-4b7f-440f-9d31-a9eec88935f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:52.427391 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be48837-4b7f-440f-9d31-a9eec88935f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:52.429335 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be48837-4b7f-440f-9d31-a9eec88935f9 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:52.444345 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 496/985] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:52.453983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b93e7-e296-43ae-a02e-39647af611f7 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:52.454762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be48837-4b7f-440f-9d31-a9eec88935f9 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:52.509039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cee1f7-e28a-448e-a097-773478499a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:52.509741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cee1f7-e28a-448e-a097-773478499a4d None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:52.510024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cee1f7-e28a-448e-a097-773478499a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:52.510784 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cee1f7-e28a-448e-a097-773478499a4d None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:52.511415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cee1f7-e28a-448e-a097-773478499a4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:52.515519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cee1f7-e28a-448e-a097-773478499a4d None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:52.533087 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 490/986] 10.222.0.88 () {56 vars in 1087 bytes} [Tue Aug 23 22:10:52 2022] GET /identity/v3/services => generated 1727 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:10:52.621526 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be48837-4b7f-440f-9d31-a9eec88935f9 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:52.681894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:52.693724 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54239a70-560f-40c6-9196-bb598f606ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:52.694388 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54239a70-560f-40c6-9196-bb598f606ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:52.694707 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54239a70-560f-40c6-9196-bb598f606ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:52.695020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54239a70-560f-40c6-9196-bb598f606ca7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:52.703930 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54239a70-560f-40c6-9196-bb598f606ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:52.705509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54239a70-560f-40c6-9196-bb598f606ca7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:52.726689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-54239a70-560f-40c6-9196-bb598f606ca7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:52.726689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:52.726689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:52.726689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:52.726689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:52.726689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:52.726689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:52.726689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:52.726689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:52.726689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:52.726689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:52.726689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:52.726689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:52.726689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:52.726689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:52.728668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:52.730700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:52.730700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:52.730700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:52.730700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:52.730700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:52.730700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:52.730700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:52.730700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:52.730700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 497/987] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:52.739215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cee1f7-e28a-448e-a097-773478499a4d None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:52.917079 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cee1f7-e28a-448e-a097-773478499a4d None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:52.962456 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbe6c1-5328-45ba-81eb-bacc31f7f220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:52.963184 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbe6c1-5328-45ba-81eb-bacc31f7f220 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:52.963595 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbe6c1-5328-45ba-81eb-bacc31f7f220 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:52.963998 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbe6c1-5328-45ba-81eb-bacc31f7f220 None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:52.964856 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefbe6c1-5328-45ba-81eb-bacc31f7f220 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:52.965334 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefbe6c1-5328-45ba-81eb-bacc31f7f220 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:52.967842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefbe6c1-5328-45ba-81eb-bacc31f7f220 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:52.986925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 491/988] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:52.988508 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54239a70-560f-40c6-9196-bb598f606ca7 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:53.107845 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54239a70-560f-40c6-9196-bb598f606ca7 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:53.147324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca5b04-718a-46c5-b013-c0ce478173a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:53.147985 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca5b04-718a-46c5-b013-c0ce478173a8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:53.148269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca5b04-718a-46c5-b013-c0ce478173a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:53.148565 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca5b04-718a-46c5-b013-c0ce478173a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:53.149411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca5b04-718a-46c5-b013-c0ce478173a8 None admin] RBAC: Adding query filter params (role.id=97c55e5f00404a29bb3f2df9566b6b0f, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=cb001b14eec54047aee42ca6bd4a1eba) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:53.149726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca5b04-718a-46c5-b013-c0ce478173a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:53.151161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca5b04-718a-46c5-b013-c0ce478173a8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:53.165422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 498/989] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:52 2022] GET /identity/v3/role_assignments?user.id=cb001b14eec54047aee42ca6bd4a1eba&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=97c55e5f00404a29bb3f2df9566b6b0f => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:54.615178 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b13954-9063-4d90-a78c-9043b1515e48 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:54.615787 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b13954-9063-4d90-a78c-9043b1515e48 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:54.615787 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b13954-9063-4d90-a78c-9043b1515e48 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:54.617368 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 492/990] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:54.623436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cedb63-bc69-4611-a12c-788c27001b58 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:54.623962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cedb63-bc69-4611-a12c-788c27001b58 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:54.623962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cedb63-bc69-4611-a12c-788c27001b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:54.644119 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-29cedb63-bc69-4611-a12c-788c27001b58 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:54.645368 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-29cedb63-bc69-4611-a12c-788c27001b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:54.709327 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:54.737306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 499/991] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:55.017873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b13954-9063-4d90-a78c-9043b1515e48 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:55.021144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:55.037310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237dbd45-1733-4be4-b051-f6996006b223 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.037763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237dbd45-1733-4be4-b051-f6996006b223 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.038375 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237dbd45-1733-4be4-b051-f6996006b223 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.040154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 500/992] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:55.049147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e967a8-df89-401b-9cfc-7673d7d19f15 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.049606 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e967a8-df89-401b-9cfc-7673d7d19f15 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.050060 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e967a8-df89-401b-9cfc-7673d7d19f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.075489 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-99e967a8-df89-401b-9cfc-7673d7d19f15 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:55.077935 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-99e967a8-df89-401b-9cfc-7673d7d19f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:55.177117 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b13954-9063-4d90-a78c-9043b1515e48 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:55.180108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 501/993] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:55 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:10:55.244822 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df28db-430a-4e46-adc7-21c51bfc190b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:55.245891 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df28db-430a-4e46-adc7-21c51bfc190b None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.246335 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df28db-430a-4e46-adc7-21c51bfc190b None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.246758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df28db-430a-4e46-adc7-21c51bfc190b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.248561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df28db-430a-4e46-adc7-21c51bfc190b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:55.251149 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df28db-430a-4e46-adc7-21c51bfc190b None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-d2df28db-430a-4e46-adc7-21c51bfc190b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:55.267428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:55.268653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:55.268653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:55.268653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:55.268653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:55.268653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:55.268653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:55.268653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:55.268653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:55.268653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:55.268653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:55.268653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:55.268653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:55.268653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:55.268653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:55.268653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:55.268653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:55.269955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:55.269955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:55.269955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:55.269955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:55.269955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:55.269955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:55.269955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:55.269955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:55.269955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:55.269955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:55.269955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:55.269955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:55.269955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:10:55.269955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:55.273510 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 493/994] 10.222.0.88 () {56 vars in 1126 bytes} [Tue Aug 23 22:10:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:55.278294 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e967a8-df89-401b-9cfc-7673d7d19f15 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:55.483730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e967a8-df89-401b-9cfc-7673d7d19f15 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:55.510730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df28db-430a-4e46-adc7-21c51bfc190b None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:55.519623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:55.553014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa164afe-c271-4f25-9bc3-8a92ef8ec9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:55.554224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa164afe-c271-4f25-9bc3-8a92ef8ec9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.554753 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa164afe-c271-4f25-9bc3-8a92ef8ec9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.555232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa164afe-c271-4f25-9bc3-8a92ef8ec9b8 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.556806 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa164afe-c271-4f25-9bc3-8a92ef8ec9b8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:55.557374 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa164afe-c271-4f25-9bc3-8a92ef8ec9b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:55.560049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa164afe-c271-4f25-9bc3-8a92ef8ec9b8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:55.570664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 502/995] 10.222.0.88 () {56 vars in 1122 bytes} [Tue Aug 23 22:10:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:55.575200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360e007-f5cb-4442-8c76-5cff5afceaab None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.575954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360e007-f5cb-4442-8c76-5cff5afceaab None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.576945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360e007-f5cb-4442-8c76-5cff5afceaab None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.579659 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 503/996] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:55.580929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360e007-f5cb-4442-8c76-5cff5afceaab None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:55.592567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:55.630018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df28db-430a-4e46-adc7-21c51bfc190b None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:55.665001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f74e8-60cb-4211-90e4-03ebf9bab4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:55.665671 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f74e8-60cb-4211-90e4-03ebf9bab4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.665954 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f74e8-60cb-4211-90e4-03ebf9bab4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.666227 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f74e8-60cb-4211-90e4-03ebf9bab4b6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.666846 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f74e8-60cb-4211-90e4-03ebf9bab4b6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:55.668242 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f74e8-60cb-4211-90e4-03ebf9bab4b6 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-631f74e8-60cb-4211-90e4-03ebf9bab4b6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:55.678611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:55.680580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:55.680580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:55.680580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:55.680580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:55.680580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:55.680580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:55.680580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:55.680580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:55.680580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:55.680580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:55.680580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:55.680580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:55.680580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:55.680580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:55.680580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:55.680580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:55.682426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:55.682426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:55.682426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:55.682426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:55.682426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:55.682426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:55.682426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:55.682426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:55.682426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:55.682426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:55.682426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:55.682426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:55.682426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:10:55.682426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:55.682426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 494/997] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:55.684101 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538aeed-71ca-4994-b5a3-dac34fba7261 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.684101 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538aeed-71ca-4994-b5a3-dac34fba7261 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.684101 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538aeed-71ca-4994-b5a3-dac34fba7261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.702803 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-4538aeed-71ca-4994-b5a3-dac34fba7261 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:55.704017 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4538aeed-71ca-4994-b5a3-dac34fba7261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:55.731599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360e007-f5cb-4442-8c76-5cff5afceaab None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:55.777919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 495/998] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:55 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:55.778393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538aeed-71ca-4994-b5a3-dac34fba7261 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:55.778655 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7318d91-5df6-47c4-8fe8-a0da6339c5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:55.779406 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7318d91-5df6-47c4-8fe8-a0da6339c5af None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.779684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7318d91-5df6-47c4-8fe8-a0da6339c5af None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.779954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7318d91-5df6-47c4-8fe8-a0da6339c5af None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.780606 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7318d91-5df6-47c4-8fe8-a0da6339c5af None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:55.780905 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7318d91-5df6-47c4-8fe8-a0da6339c5af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:55.782447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7318d91-5df6-47c4-8fe8-a0da6339c5af None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:55.790021 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 504/999] 10.222.0.88 () {56 vars in 1122 bytes} [Tue Aug 23 22:10:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:55.797873 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7318d91-5df6-47c4-8fe8-a0da6339c5af None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:55.851906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:55.862524 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:55.927147 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538aeed-71ca-4994-b5a3-dac34fba7261 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:55.934618 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7318d91-5df6-47c4-8fe8-a0da6339c5af None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:55.980459 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331245f-1eab-46e8-a64f-1f73d733445f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:55.981108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331245f-1eab-46e8-a64f-1f73d733445f None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.981388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331245f-1eab-46e8-a64f-1f73d733445f None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.981714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331245f-1eab-46e8-a64f-1f73d733445f None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.982845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331245f-1eab-46e8-a64f-1f73d733445f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:55.983224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331245f-1eab-46e8-a64f-1f73d733445f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:55.984487 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331245f-1eab-46e8-a64f-1f73d733445f None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:55.986580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd598a-72f8-4479-bc7f-730a98298f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:55.987611 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd598a-72f8-4479-bc7f-730a98298f61 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.988104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd598a-72f8-4479-bc7f-730a98298f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.988614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd598a-72f8-4479-bc7f-730a98298f61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.989919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd598a-72f8-4479-bc7f-730a98298f61 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:55.993134 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd598a-72f8-4479-bc7f-730a98298f61 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:55.995725 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 496/1000] 10.222.0.88 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:55.996466 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331245f-1eab-46e8-a64f-1f73d733445f None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:56.025104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 505/1001] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:10:55 2022] POST /identity/v3/endpoints => generated 325 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:56.063684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd598a-72f8-4479-bc7f-730a98298f61 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:56.154836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331245f-1eab-46e8-a64f-1f73d733445f None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:56.192276 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:56.213164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:56.220935 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd598a-72f8-4479-bc7f-730a98298f61 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:56.221350 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf82a7-ad68-46be-a47d-48b96076c863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:56.222295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf82a7-ad68-46be-a47d-48b96076c863 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:56.222739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf82a7-ad68-46be-a47d-48b96076c863 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:56.223187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf82a7-ad68-46be-a47d-48b96076c863 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:56.224142 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf82a7-ad68-46be-a47d-48b96076c863 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:56.224640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf82a7-ad68-46be-a47d-48b96076c863 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:56.226700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf82a7-ad68-46be-a47d-48b96076c863 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:56.236491 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 497/1002] 10.222.0.88 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:56.245765 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf82a7-ad68-46be-a47d-48b96076c863 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:56.285013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542dddd-ae4a-4dbf-aed8-08784ad5278e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:56.286093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542dddd-ae4a-4dbf-aed8-08784ad5278e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:56.286550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542dddd-ae4a-4dbf-aed8-08784ad5278e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:56.286986 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542dddd-ae4a-4dbf-aed8-08784ad5278e None admin] PATH_INFO: `/v3/services/image` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:56.288055 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542dddd-ae4a-4dbf-aed8-08784ad5278e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:56.290274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542dddd-ae4a-4dbf-aed8-08784ad5278e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-a542dddd-ae4a-4dbf-aed8-08784ad5278e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:56.305675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:56.306843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:56.306843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:56.306843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:56.306843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:56.306843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:56.306843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:56.306843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:56.306843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:56.306843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:56.306843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:56.306843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:56.306843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:56.306843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:56.306843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:56.306843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:56.306843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:56.307738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:56.307738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:56.307738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:56.307738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:56.307738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:56.307738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:56.307738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:56.307738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:56.307738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:56.307738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:56.307738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:56.307738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:56.307738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:10:56.307738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:10:56.308539 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 506/1003] 10.222.0.88 () {56 vars in 1105 bytes} [Tue Aug 23 22:10:56 2022] GET /identity/v3/services/image => generated 86 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:56.317048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542dddd-ae4a-4dbf-aed8-08784ad5278e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:56.422125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf82a7-ad68-46be-a47d-48b96076c863 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:56.469684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b9da5-5df0-4798-8887-bf514df75fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:56.470733 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b9da5-5df0-4798-8887-bf514df75fca None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:56.471227 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b9da5-5df0-4798-8887-bf514df75fca None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:56.471661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b9da5-5df0-4798-8887-bf514df75fca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:56.472879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b9da5-5df0-4798-8887-bf514df75fca None admin] RBAC: Adding query filter params (interface=public, service_id=3435f7f2b7b54145a5782fa76146017a, region_id=RegionOne) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:56.473401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b9da5-5df0-4798-8887-bf514df75fca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:56.476041 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b9da5-5df0-4798-8887-bf514df75fca None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:56.481556 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542dddd-ae4a-4dbf-aed8-08784ad5278e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:56.490749 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 498/1004] 10.222.0.88 () {56 vars in 1251 bytes} [Tue Aug 23 22:10:56 2022] GET /identity/v3/endpoints?service_id=3435f7f2b7b54145a5782fa76146017a&interface=public®ion_id=RegionOne => generated 194 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:56.492814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b9da5-5df0-4798-8887-bf514df75fca None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:56.531496 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5acf4c-f52a-4f58-a68b-0bfc95961d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:56.532415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5acf4c-f52a-4f58-a68b-0bfc95961d80 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:56.532914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5acf4c-f52a-4f58-a68b-0bfc95961d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:56.533445 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5acf4c-f52a-4f58-a68b-0bfc95961d80 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:56.534404 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5acf4c-f52a-4f58-a68b-0bfc95961d80 None admin] RBAC: Adding query filter params (name=image) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:56.534950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5acf4c-f52a-4f58-a68b-0bfc95961d80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:56.537485 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5acf4c-f52a-4f58-a68b-0bfc95961d80 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:56.549491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 507/1005] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:56.555016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5acf4c-f52a-4f58-a68b-0bfc95961d80 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:56.563965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:56.589373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:56.662063 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b9da5-5df0-4798-8887-bf514df75fca None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:56.672854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5acf4c-f52a-4f58-a68b-0bfc95961d80 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:56.703653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb3a98-b82b-412b-a4f2-6383a0bbcd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:56.704370 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb3a98-b82b-412b-a4f2-6383a0bbcd46 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:56.704660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb3a98-b82b-412b-a4f2-6383a0bbcd46 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:56.704931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb3a98-b82b-412b-a4f2-6383a0bbcd46 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:56.705604 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb3a98-b82b-412b-a4f2-6383a0bbcd46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:56.707118 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb3a98-b82b-412b-a4f2-6383a0bbcd46 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:56.713468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae8981-587e-46a8-8508-40ef8a6aab6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:56.714135 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae8981-587e-46a8-8508-40ef8a6aab6e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:56.725379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae8981-587e-46a8-8508-40ef8a6aab6e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:56.725544 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 499/1006] 10.222.0.88 () {56 vars in 1087 bytes} [Tue Aug 23 22:10:56 2022] GET /identity/v3/services => generated 1727 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:10:56.725681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae8981-587e-46a8-8508-40ef8a6aab6e None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:56.725681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae8981-587e-46a8-8508-40ef8a6aab6e None admin] RBAC: Adding query filter params (type=image) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:56.725681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae8981-587e-46a8-8508-40ef8a6aab6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:56.725681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae8981-587e-46a8-8508-40ef8a6aab6e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:56.729724 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 508/1007] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:56 2022] GET /identity/v3/services?type=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:58.121524 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eef61b-a8ef-4cca-89d3-1e2e8d1da689 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.122296 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eef61b-a8ef-4cca-89d3-1e2e8d1da689 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.122296 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eef61b-a8ef-4cca-89d3-1e2e8d1da689 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.123548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 500/1008] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:58.130634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f171f-cf16-4951-a414-b578f3af582d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.131165 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f171f-cf16-4951-a414-b578f3af582d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.131743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f171f-cf16-4951-a414-b578f3af582d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.151756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ce4f171f-cf16-4951-a414-b578f3af582d None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:58.152950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ce4f171f-cf16-4951-a414-b578f3af582d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:58.251149 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 509/1009] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:10:58.650123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eef61b-a8ef-4cca-89d3-1e2e8d1da689 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:58.761481 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eef61b-a8ef-4cca-89d3-1e2e8d1da689 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:58.781613 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:58.807890 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd480320-33f9-48ff-9cd8-b7e5a97e1e8d None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.808184 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd480320-33f9-48ff-9cd8-b7e5a97e1e8d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.808516 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd480320-33f9-48ff-9cd8-b7e5a97e1e8d None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.810036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 510/1010] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:58.816265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b52f2-6965-44b8-bd4b-c383769cccfb None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.816564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b52f2-6965-44b8-bd4b-c383769cccfb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.816842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b52f2-6965-44b8-bd4b-c383769cccfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.819103 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147e6d1-0e19-426d-87b2-e76220a8e520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:58.819748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147e6d1-0e19-426d-87b2-e76220a8e520 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.820067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147e6d1-0e19-426d-87b2-e76220a8e520 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.820292 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147e6d1-0e19-426d-87b2-e76220a8e520 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.821499 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147e6d1-0e19-426d-87b2-e76220a8e520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:58.823069 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147e6d1-0e19-426d-87b2-e76220a8e520 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:58.829111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 501/1011] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:58.832818 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147e6d1-0e19-426d-87b2-e76220a8e520 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:58.837491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a29b52f2-6965-44b8-bd4b-c383769cccfb None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:58.838788 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a29b52f2-6965-44b8-bd4b-c383769cccfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:58.864401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:58.960382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 511/1012] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:10:58.962142 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5cd4b-c110-4f1a-9e52-6cbc76baf3cb None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.962581 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5cd4b-c110-4f1a-9e52-6cbc76baf3cb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.962991 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5cd4b-c110-4f1a-9e52-6cbc76baf3cb None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.964894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 512/1013] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:10:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:58.971994 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c260f7-4a96-404f-b215-55a5dd510930 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.972444 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c260f7-4a96-404f-b215-55a5dd510930 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.973095 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c260f7-4a96-404f-b215-55a5dd510930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.994527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e3c260f7-4a96-404f-b215-55a5dd510930 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:58.995782 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e3c260f7-4a96-404f-b215-55a5dd510930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:59.008398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147e6d1-0e19-426d-87b2-e76220a8e520 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:59.054482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc3c1a-50da-4809-a39b-acafa2ae0183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:59.055416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc3c1a-50da-4809-a39b-acafa2ae0183 None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:59.055826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc3c1a-50da-4809-a39b-acafa2ae0183 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:59.056208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc3c1a-50da-4809-a39b-acafa2ae0183 None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:59.056885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc3c1a-50da-4809-a39b-acafa2ae0183 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:59.058378 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc3c1a-50da-4809-a39b-acafa2ae0183 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:59.090287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 502/1014] 10.222.0.88 () {60 vars in 1141 bytes} [Tue Aug 23 22:10:58 2022] POST /identity/v3/projects => generated 311 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:59.094344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 513/1015] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:10:59.378470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc3c1a-50da-4809-a39b-acafa2ae0183 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:59.418137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:59.477543 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c260f7-4a96-404f-b215-55a5dd510930 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:59.501713 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc3c1a-50da-4809-a39b-acafa2ae0183 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:59.542051 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483cae1-d674-4870-85e4-fff78949baa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:59.542704 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483cae1-d674-4870-85e4-fff78949baa7 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:59.542987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483cae1-d674-4870-85e4-fff78949baa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:59.543260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483cae1-d674-4870-85e4-fff78949baa7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:59.543860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483cae1-d674-4870-85e4-fff78949baa7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:59.546384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483cae1-d674-4870-85e4-fff78949baa7 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-9483cae1-d674-4870-85e4-fff78949baa7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:59.556677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:59.558046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:59.558046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:59.558046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:59.558046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:59.558046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:59.558046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:59.558046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:59.558046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:59.558046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:59.558046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:59.558046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:59.558046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:59.558046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:59.558046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:59.558046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:59.558046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:59.558897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:59.558897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:59.558897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:59.558897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:59.558897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:59.558897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:59.558897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:59.558897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:59.558897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:59.558897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:59.558897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:59.558897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:59.558897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:10:59.558897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:10:59.558897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 503/1016] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:59.565354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483cae1-d674-4870-85e4-fff78949baa7 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:59.621251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:59.652038 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c260f7-4a96-404f-b215-55a5dd510930 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:59.695636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ce9ad-560f-48f0-bb86-6644921c996f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:59.696269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ce9ad-560f-48f0-bb86-6644921c996f None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:59.696551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ce9ad-560f-48f0-bb86-6644921c996f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:59.696826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ce9ad-560f-48f0-bb86-6644921c996f None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:59.697544 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132ce9ad-560f-48f0-bb86-6644921c996f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:59.698994 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132ce9ad-560f-48f0-bb86-6644921c996f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:59.711299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:59.720723 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483cae1-d674-4870-85e4-fff78949baa7 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:59.725120 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 514/1017] 10.222.0.88 () {60 vars in 1141 bytes} [Tue Aug 23 22:10:59 2022] POST /identity/v3/services => generated 242 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:59.790159 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7875412-ae93-49d7-adfb-8508231e3fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:59.791177 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7875412-ae93-49d7-adfb-8508231e3fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:59.791609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7875412-ae93-49d7-adfb-8508231e3fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:59.792047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7875412-ae93-49d7-adfb-8508231e3fb2 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:59.793171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7875412-ae93-49d7-adfb-8508231e3fb2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:59.796398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7875412-ae93-49d7-adfb-8508231e3fb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:59.796740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7875412-ae93-49d7-adfb-8508231e3fb2 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:59.811461 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 504/1018] 10.222.0.88 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:59.813438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ce9ad-560f-48f0-bb86-6644921c996f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:00.007684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ce9ad-560f-48f0-bb86-6644921c996f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:00.077196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1596250-06e3-4185-ae12-f9bef76430ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:00.078233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1596250-06e3-4185-ae12-f9bef76430ec None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:00.078715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1596250-06e3-4185-ae12-f9bef76430ec None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:00.079197 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1596250-06e3-4185-ae12-f9bef76430ec None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:00.080346 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1596250-06e3-4185-ae12-f9bef76430ec None admin] RBAC: Adding query filter params (type=compute) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:00.080890 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1596250-06e3-4185-ae12-f9bef76430ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:00.083458 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1596250-06e3-4185-ae12-f9bef76430ec None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:00.093322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 515/1019] 10.222.0.88 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:00.103824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7875412-ae93-49d7-adfb-8508231e3fb2 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:00.303877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7875412-ae93-49d7-adfb-8508231e3fb2 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:00.348690 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ae9ce-1853-4319-bf05-ebffedcc423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:00.349410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ae9ce-1853-4319-bf05-ebffedcc423a None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:00.349699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ae9ce-1853-4319-bf05-ebffedcc423a None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:00.349968 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ae9ce-1853-4319-bf05-ebffedcc423a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:00.350671 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ae9ce-1853-4319-bf05-ebffedcc423a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:00.352202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ae9ce-1853-4319-bf05-ebffedcc423a None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:00.387128 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 505/1020] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:11:00 2022] POST /identity/v3/endpoints => generated 334 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:01.913816 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d94c44-5312-488a-b570-cb2aa9abc2e4 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:01.914595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d94c44-5312-488a-b570-cb2aa9abc2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:01.914595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d94c44-5312-488a-b570-cb2aa9abc2e4 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:01.915999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 516/1021] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:11:01.922467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96178e-73d5-411f-9e19-bab78b45361e None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:01.923241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96178e-73d5-411f-9e19-bab78b45361e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:01.923241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96178e-73d5-411f-9e19-bab78b45361e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:01.943510 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-6e96178e-73d5-411f-9e19-bab78b45361e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:01.944789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6e96178e-73d5-411f-9e19-bab78b45361e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:01.955744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:01.997923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765795f-2f41-40a4-aab3-b6bb38857c5f None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:01.998202 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765795f-2f41-40a4-aab3-b6bb38857c5f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:01.998504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765795f-2f41-40a4-aab3-b6bb38857c5f None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:02.000987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 517/1022] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:02.009758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbe8a3-c17a-4c27-adf7-09bbf8b66858 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:02.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbe8a3-c17a-4c27-adf7-09bbf8b66858 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:02.011408 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbe8a3-c17a-4c27-adf7-09bbf8b66858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:02.030626 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:02.036729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-35bbe8a3-c17a-4c27-adf7-09bbf8b66858 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:02.038699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-35bbe8a3-c17a-4c27-adf7-09bbf8b66858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:02.045326 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 506/1023] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:02.124415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 518/1024] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:02.319005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96178e-73d5-411f-9e19-bab78b45361e None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:02.443965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96178e-73d5-411f-9e19-bab78b45361e None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:02.486684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a60d3e-b5be-48ef-b965-0e0bf5d5a65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:02.487380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a60d3e-b5be-48ef-b965-0e0bf5d5a65e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:02.487658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a60d3e-b5be-48ef-b965-0e0bf5d5a65e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:02.487941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a60d3e-b5be-48ef-b965-0e0bf5d5a65e None admin] PATH_INFO: `/v3/services/image` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:02.488518 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a60d3e-b5be-48ef-b965-0e0bf5d5a65e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:02.490021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a60d3e-b5be-48ef-b965-0e0bf5d5a65e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-73a60d3e-b5be-48ef-b965-0e0bf5d5a65e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:02.501346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:02.503253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:02.503253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:02.503253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:02.503253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:02.503253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:02.503253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:02.503253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:02.503253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:02.503253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:02.503253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:02.503253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:02.503253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:02.503253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:02.503253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:02.503253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:02.503253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:02.505407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:02.505407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:02.505407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:02.505407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:02.505407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:02.505407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:02.505407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:02.505407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:02.505407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:02.505407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:02.505407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:02.505407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:02.505407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:11:02.505407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:11:02.508021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 507/1025] 10.222.0.88 () {56 vars in 1105 bytes} [Tue Aug 23 22:11:02 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:02.509255 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbe8a3-c17a-4c27-adf7-09bbf8b66858 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:02.565980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a60d3e-b5be-48ef-b965-0e0bf5d5a65e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:02.571277 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:02.662361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbe8a3-c17a-4c27-adf7-09bbf8b66858 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:02.705602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2576e-b428-4941-be41-6edda4750384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:02.706125 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2576e-b428-4941-be41-6edda4750384 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:02.706406 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2576e-b428-4941-be41-6edda4750384 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:02.706684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2576e-b428-4941-be41-6edda4750384 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:02.707314 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2576e-b428-4941-be41-6edda4750384 None admin] RBAC: Adding query filter params (name=image) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:02.707610 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2576e-b428-4941-be41-6edda4750384 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:02.708996 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2576e-b428-4941-be41-6edda4750384 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:02.711710 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:02.717048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 519/1026] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:02.725450 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2576e-b428-4941-be41-6edda4750384 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:02.744541 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a60d3e-b5be-48ef-b965-0e0bf5d5a65e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:02.812824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb7c70-90bf-42eb-b8fd-2482645b047f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:02.813993 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb7c70-90bf-42eb-b8fd-2482645b047f None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:02.814446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb7c70-90bf-42eb-b8fd-2482645b047f None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:02.814881 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb7c70-90bf-42eb-b8fd-2482645b047f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:02.817135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb7c70-90bf-42eb-b8fd-2482645b047f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:02.820166 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb7c70-90bf-42eb-b8fd-2482645b047f None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:02.825102 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 508/1027] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:02.832906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb7c70-90bf-42eb-b8fd-2482645b047f None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:02.871149 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2576e-b428-4941-be41-6edda4750384 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:02.918093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e30ddf-0a24-4695-aaba-f0f45235564f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:02.919140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e30ddf-0a24-4695-aaba-f0f45235564f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:02.919832 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e30ddf-0a24-4695-aaba-f0f45235564f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:02.920477 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e30ddf-0a24-4695-aaba-f0f45235564f None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:02.920998 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e30ddf-0a24-4695-aaba-f0f45235564f None admin] RBAC: Adding query filter params (type=image) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:02.921612 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e30ddf-0a24-4695-aaba-f0f45235564f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:02.923200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e30ddf-0a24-4695-aaba-f0f45235564f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:02.937113 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 520/1028] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:02.937371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:02.941188 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e30ddf-0a24-4695-aaba-f0f45235564f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:02.979513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb7c70-90bf-42eb-b8fd-2482645b047f None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:03.025799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f796e75-703a-4318-bef9-c394a43a7ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:03.026439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f796e75-703a-4318-bef9-c394a43a7ff2 None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:03.026720 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f796e75-703a-4318-bef9-c394a43a7ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:03.026997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f796e75-703a-4318-bef9-c394a43a7ff2 None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:03.027678 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f796e75-703a-4318-bef9-c394a43a7ff2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:03.029379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:03.030658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f796e75-703a-4318-bef9-c394a43a7ff2 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:03.061535 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 509/1029] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:11:02 2022] POST /identity/v3/users => generated 285 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:11:03.089504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e30ddf-0a24-4695-aaba-f0f45235564f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:03.136379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9d2f2-41a6-4814-87bf-b90b07c5c9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:03.137274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9d2f2-41a6-4814-87bf-b90b07c5c9a5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:03.137695 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9d2f2-41a6-4814-87bf-b90b07c5c9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:03.138702 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9d2f2-41a6-4814-87bf-b90b07c5c9a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:03.139594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9d2f2-41a6-4814-87bf-b90b07c5c9a5 None admin] RBAC: Adding query filter params (interface=public, service_id=07ce254030ab43a6ba83407fde136d76, region_id=RegionOne) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:03.140058 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9d2f2-41a6-4814-87bf-b90b07c5c9a5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:03.141787 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9d2f2-41a6-4814-87bf-b90b07c5c9a5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:03.149319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 521/1030] 10.222.0.88 () {56 vars in 1251 bytes} [Tue Aug 23 22:11:02 2022] GET /identity/v3/endpoints?service_id=07ce254030ab43a6ba83407fde136d76&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:03.156140 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f796e75-703a-4318-bef9-c394a43a7ff2 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:03.320874 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f796e75-703a-4318-bef9-c394a43a7ff2 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:03.356505 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:03.368171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f4d80-c514-46bc-a1a0-08c5110c35f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:03.368791 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f4d80-c514-46bc-a1a0-08c5110c35f8 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:03.369104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f4d80-c514-46bc-a1a0-08c5110c35f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:03.369422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f4d80-c514-46bc-a1a0-08c5110c35f8 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:03.369972 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f4d80-c514-46bc-a1a0-08c5110c35f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:03.371374 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f4d80-c514-46bc-a1a0-08c5110c35f8 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:03.389092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 510/1031] 10.222.0.88 () {56 vars in 1087 bytes} [Tue Aug 23 22:11:03 2022] GET /identity/v3/services => generated 1957 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:11:03.617570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea6e46-1840-4c73-a770-9240089ff4d9 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:03.617997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea6e46-1840-4c73-a770-9240089ff4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:03.618460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea6e46-1840-4c73-a770-9240089ff4d9 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:03.621443 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 522/1032] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:03.631161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6239137-db9e-4683-adb1-a62097a10b50 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:03.631665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6239137-db9e-4683-adb1-a62097a10b50 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:03.632130 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6239137-db9e-4683-adb1-a62097a10b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:03.664174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-e6239137-db9e-4683-adb1-a62097a10b50 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:03.666242 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e6239137-db9e-4683-adb1-a62097a10b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:03.801635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 511/1033] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:04.250747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea6e46-1840-4c73-a770-9240089ff4d9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:04.390016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea6e46-1840-4c73-a770-9240089ff4d9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:04.425988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:04.438319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b267d-3d1d-4257-887e-7477939a66b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:04.439380 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b267d-3d1d-4257-887e-7477939a66b4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:04.439854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b267d-3d1d-4257-887e-7477939a66b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:04.440322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b267d-3d1d-4257-887e-7477939a66b4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:04.454597 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b267d-3d1d-4257-887e-7477939a66b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:04.459558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-7c3b267d-3d1d-4257-887e-7477939a66b4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:04.459558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:04.459558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:04.459558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:04.459558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:04.459558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:04.459558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:04.459558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:04.459558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:04.459558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:04.459558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:04.459558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:04.459558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:04.459558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:04.459558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:04.459558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:04.461504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:04.461504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:04.461504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:04.461504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:04.461504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:04.461504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:04.461504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:04.461504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:04.461504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:04.461504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:04.461504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:04.461504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:04.461504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:04.461504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:04.461504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:04.461504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:04.463319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:04.463319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:04.463319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:04.463319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:04.463319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:04.463319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:04.463319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:04.463319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:04.463319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:04.463319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:04.463319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:04.463319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:04.465175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 523/1034] 10.222.0.88 () {56 vars in 1120 bytes} [Tue Aug 23 22:11:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:04.467461 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6239137-db9e-4683-adb1-a62097a10b50 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:04.587044 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6239137-db9e-4683-adb1-a62097a10b50 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:04.612820 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:04.628522 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb6a80-e1f3-4585-b7c4-3e4115a50365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:04.629161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb6a80-e1f3-4585-b7c4-3e4115a50365 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:04.629439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb6a80-e1f3-4585-b7c4-3e4115a50365 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:04.629705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb6a80-e1f3-4585-b7c4-3e4115a50365 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:04.630376 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb6a80-e1f3-4585-b7c4-3e4115a50365 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:04.630670 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb6a80-e1f3-4585-b7c4-3e4115a50365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:04.632044 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb6a80-e1f3-4585-b7c4-3e4115a50365 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:04.644689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 512/1035] 10.222.0.88 () {56 vars in 1115 bytes} [Tue Aug 23 22:11:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:04.647721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b267d-3d1d-4257-887e-7477939a66b4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:04.782230 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b267d-3d1d-4257-887e-7477939a66b4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:04.846104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440554da-c2cc-402e-a676-71a4f5c7d24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:04.847080 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440554da-c2cc-402e-a676-71a4f5c7d24a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:04.847596 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440554da-c2cc-402e-a676-71a4f5c7d24a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:04.848086 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440554da-c2cc-402e-a676-71a4f5c7d24a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:04.863375 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440554da-c2cc-402e-a676-71a4f5c7d24a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:04.864863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440554da-c2cc-402e-a676-71a4f5c7d24a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:04.884240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-440554da-c2cc-402e-a676-71a4f5c7d24a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:04.884240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:04.884240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:04.884240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:04.884240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:04.884240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:04.884240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:04.884240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:04.884240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:04.884240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:04.884240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:04.884240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:04.884240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:04.884240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:04.884240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:04.884240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:04.893803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:04.895795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:04.895795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:04.895795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:04.895795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 524/1036] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:04.899761 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb6a80-e1f3-4585-b7c4-3e4115a50365 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:05.049504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb6a80-e1f3-4585-b7c4-3e4115a50365 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:05.116154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ca567-7c32-4354-b51f-2a796aa332ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:05.117288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ca567-7c32-4354-b51f-2a796aa332ab None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:05.117661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ca567-7c32-4354-b51f-2a796aa332ab None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:05.118163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ca567-7c32-4354-b51f-2a796aa332ab None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:05.119401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ca567-7c32-4354-b51f-2a796aa332ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:05.119956 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ca567-7c32-4354-b51f-2a796aa332ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:05.123340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ca567-7c32-4354-b51f-2a796aa332ab None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:05.148045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 513/1037] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:05.157137 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440554da-c2cc-402e-a676-71a4f5c7d24a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:05.242616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:05.351309 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af8d95-1f5c-4320-91a8-d0f5f562cd51 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:05.351676 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af8d95-1f5c-4320-91a8-d0f5f562cd51 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:05.352092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af8d95-1f5c-4320-91a8-d0f5f562cd51 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:05.353962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 514/1038] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:05.355946 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440554da-c2cc-402e-a676-71a4f5c7d24a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:05.361585 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b883f-83d8-4d38-a9c2-7cc02ab5917f None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:05.362495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b883f-83d8-4d38-a9c2-7cc02ab5917f None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:05.362886 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b883f-83d8-4d38-a9c2-7cc02ab5917f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:05.385980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-142b883f-83d8-4d38-a9c2-7cc02ab5917f None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:05.387496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-142b883f-83d8-4d38-a9c2-7cc02ab5917f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:05.418860 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc54878-d3e7-435c-b725-6c5381053c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:05.420646 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc54878-d3e7-435c-b725-6c5381053c97 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:05.421194 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc54878-d3e7-435c-b725-6c5381053c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:05.421673 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc54878-d3e7-435c-b725-6c5381053c97 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:05.437436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc54878-d3e7-435c-b725-6c5381053c97 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:05.440129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc54878-d3e7-435c-b725-6c5381053c97 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:05.448935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:05.463602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-ebc54878-d3e7-435c-b725-6c5381053c97 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:11:05.463602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:05.463602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:05.463602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:05.463602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:05.463602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:05.463602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:05.463602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:05.463602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:05.463602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:05.463602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:05.463602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:05.463602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:11:05.463602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:11:05.463602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:11:05.463602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:05.465650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:05.469271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:05.469271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:05.469271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:11:05.469271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:11:05.469271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:11:05.469271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:11:05.469271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:11:05.469271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:11:05.469271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:11:05.469271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:05.473523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 525/1039] 10.222.0.88 () {56 vars in 1093 bytes} [Tue Aug 23 22:11:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:05.475370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc54878-d3e7-435c-b725-6c5381053c97 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:05.527437 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 515/1040] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:05.663223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:05.669564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc54878-d3e7-435c-b725-6c5381053c97 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:05.728032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90a31c-b133-4ced-958a-7d4569d0b4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:05.729000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90a31c-b133-4ced-958a-7d4569d0b4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:05.729531 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90a31c-b133-4ced-958a-7d4569d0b4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:05.730009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90a31c-b133-4ced-958a-7d4569d0b4f5 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:05.730983 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90a31c-b133-4ced-958a-7d4569d0b4f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:05.731485 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90a31c-b133-4ced-958a-7d4569d0b4f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:05.734360 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90a31c-b133-4ced-958a-7d4569d0b4f5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:05.769334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 526/1041] 10.222.0.88 () {56 vars in 1133 bytes} [Tue Aug 23 22:11:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:05.775674 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b883f-83d8-4d38-a9c2-7cc02ab5917f None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:05.831568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fc5f0-b0af-4fbc-a1f4-c4eec1508b37 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:05.832041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fc5f0-b0af-4fbc-a1f4-c4eec1508b37 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:05.832413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fc5f0-b0af-4fbc-a1f4-c4eec1508b37 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:05.834531 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 527/1042] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:11:05.843760 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557e6af-ab1d-446c-b16c-7e1410d090c1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:05.844025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557e6af-ab1d-446c-b16c-7e1410d090c1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:05.844398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557e6af-ab1d-446c-b16c-7e1410d090c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:05.873681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-3557e6af-ab1d-446c-b16c-7e1410d090c1 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:05.875586 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3557e6af-ab1d-446c-b16c-7e1410d090c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:05.928217 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b883f-83d8-4d38-a9c2-7cc02ab5917f None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:05.983507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373e47b-8b5b-46b9-b976-2f72c138b0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:05.984456 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373e47b-8b5b-46b9-b976-2f72c138b0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:05.984905 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373e47b-8b5b-46b9-b976-2f72c138b0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:05.985525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373e47b-8b5b-46b9-b976-2f72c138b0e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:05.985651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 528/1043] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:05.986903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557e6af-ab1d-446c-b16c-7e1410d090c1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:06.056432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:06.097748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373e47b-8b5b-46b9-b976-2f72c138b0e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:06.099717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373e47b-8b5b-46b9-b976-2f72c138b0e3 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:06.121751 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:06.143063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557e6af-ab1d-446c-b16c-7e1410d090c1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:06.183842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd9fee-8b0e-4fc9-8a63-7b795cf8b9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:06.184388 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd9fee-8b0e-4fc9-8a63-7b795cf8b9cb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:06.185304 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd9fee-8b0e-4fc9-8a63-7b795cf8b9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:06.185593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd9fee-8b0e-4fc9-8a63-7b795cf8b9cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:06.197591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd9fee-8b0e-4fc9-8a63-7b795cf8b9cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:06.203398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-0cbd9fee-8b0e-4fc9-8a63-7b795cf8b9cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:06.203398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:06.203398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:06.203398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:06.203398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:06.203398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:06.203398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:06.203398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:06.203398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:06.203398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:06.203398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:06.203398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:06.203398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:06.203398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:06.203398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:06.203398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:06.204397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:06.204397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:06.204397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:06.204397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:06.204397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:06.204397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:06.204397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:06.204397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:06.204397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:06.204397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:06.204397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:06.204397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:06.204397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:06.204397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:06.204397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:06.204397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:06.205428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:06.205428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:06.205428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:06.205428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:06.205428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:06.205428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:06.205428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:06.205428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:06.205428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:06.205428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:06.205428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:06.205428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:06.205428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 529/1044] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:11:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:06.210841 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd9fee-8b0e-4fc9-8a63-7b795cf8b9cb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:06.245749 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 516/1045] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:05 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 22:11:06.251763 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373e47b-8b5b-46b9-b976-2f72c138b0e3 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:06.323091 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd9fee-8b0e-4fc9-8a63-7b795cf8b9cb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:06.325237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:06.357718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373e47b-8b5b-46b9-b976-2f72c138b0e3 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:06.359848 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3985cf1-c2d8-4796-9906-46edf976b2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:06.360504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3985cf1-c2d8-4796-9906-46edf976b2ad None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:06.360790 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3985cf1-c2d8-4796-9906-46edf976b2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:06.361082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3985cf1-c2d8-4796-9906-46edf976b2ad None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:06.361784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3985cf1-c2d8-4796-9906-46edf976b2ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:06.362091 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3985cf1-c2d8-4796-9906-46edf976b2ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:06.363514 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3985cf1-c2d8-4796-9906-46edf976b2ad None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:06.377460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 530/1046] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:11:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:06.377460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3985cf1-c2d8-4796-9906-46edf976b2ad None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:06.386523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:06.396704 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d40c91-0245-4633-b2c1-ddf9220f7625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:06.397384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d40c91-0245-4633-b2c1-ddf9220f7625 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:06.397668 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d40c91-0245-4633-b2c1-ddf9220f7625 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:06.397941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d40c91-0245-4633-b2c1-ddf9220f7625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:06.410325 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d40c91-0245-4633-b2c1-ddf9220f7625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:06.411792 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d40c91-0245-4633-b2c1-ddf9220f7625 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:06.431251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-f4d40c91-0245-4633-b2c1-ddf9220f7625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:06.431251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:06.431251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:06.431251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:06.431251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:06.431251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:06.431251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:06.431251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:06.431251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:06.431251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:06.431251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:06.431251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:06.431251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:06.431251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:06.431251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:06.431251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:06.432529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:06.433472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:06.433472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:06.433472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:11:06.433472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 517/1047] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:06.433472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d40c91-0245-4633-b2c1-ddf9220f7625 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:06.491928 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3985cf1-c2d8-4796-9906-46edf976b2ad None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:06.528349 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abd03a-5ed5-4cb0-80c7-8211f1995ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:06.529910 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abd03a-5ed5-4cb0-80c7-8211f1995ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:06.530233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abd03a-5ed5-4cb0-80c7-8211f1995ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:06.530506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abd03a-5ed5-4cb0-80c7-8211f1995ce8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:06.531124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abd03a-5ed5-4cb0-80c7-8211f1995ce8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:06.532490 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abd03a-5ed5-4cb0-80c7-8211f1995ce8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:06.541574 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d40c91-0245-4633-b2c1-ddf9220f7625 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-29abd03a-5ed5-4cb0-80c7-8211f1995ce8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:06.542154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:06.543224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:06.543224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:06.543224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:06.543224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:06.543224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:06.543224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:06.543224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:06.543224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:06.543224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:06.543224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:06.543224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:06.543224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:06.543224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:06.543224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:06.543224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:06.543224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:06.544182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:06.544182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:06.544182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:06.544182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:06.544182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:06.544182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:06.544182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:06.544182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:06.544182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:06.544182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:06.544182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:06.544182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:06.544182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:11:06.544182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:06.544182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 531/1048] 10.222.0.88 () {56 vars in 1105 bytes} [Tue Aug 23 22:11:06 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:06.544966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abd03a-5ed5-4cb0-80c7-8211f1995ce8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:06.570811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:06.583369 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c361a41-36e2-4c61-99a6-bc13ab36ef1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:06.584016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c361a41-36e2-4c61-99a6-bc13ab36ef1d None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:06.584287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c361a41-36e2-4c61-99a6-bc13ab36ef1d None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:06.584555 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c361a41-36e2-4c61-99a6-bc13ab36ef1d None admin] PATH_INFO: `/v3/users/40dc7e074b7946cab8a8812a655b8312` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:06.599905 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c361a41-36e2-4c61-99a6-bc13ab36ef1d None admin] RBAC: Authorizing `identity:get_user(user_id=40dc7e074b7946cab8a8812a655b8312)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:06.601430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c361a41-36e2-4c61-99a6-bc13ab36ef1d None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:06.603142 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 518/1049] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:06 2022] GET /identity/v3/users/40dc7e074b7946cab8a8812a655b8312 => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:06.603923 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c361a41-36e2-4c61-99a6-bc13ab36ef1d None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:06.634390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:06.667763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abd03a-5ed5-4cb0-80c7-8211f1995ce8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:06.706288 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a57dd-50a0-4a53-b6a1-b3e664ab4b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:06.706897 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a57dd-50a0-4a53-b6a1-b3e664ab4b44 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:06.707176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a57dd-50a0-4a53-b6a1-b3e664ab4b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:06.707453 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a57dd-50a0-4a53-b6a1-b3e664ab4b44 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:06.708116 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a57dd-50a0-4a53-b6a1-b3e664ab4b44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:06.708461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a57dd-50a0-4a53-b6a1-b3e664ab4b44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:06.709880 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a57dd-50a0-4a53-b6a1-b3e664ab4b44 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:06.722538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 532/1050] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:06.723390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a57dd-50a0-4a53-b6a1-b3e664ab4b44 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:06.743715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c361a41-36e2-4c61-99a6-bc13ab36ef1d None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:06.792402 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef886fa0-d267-475c-8f76-bbe548843615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:06.793419 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef886fa0-d267-475c-8f76-bbe548843615 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:06.793838 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef886fa0-d267-475c-8f76-bbe548843615 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:06.794287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef886fa0-d267-475c-8f76-bbe548843615 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:06.795322 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef886fa0-d267-475c-8f76-bbe548843615 None admin] RBAC: Adding query filter params (name=image) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:06.795831 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef886fa0-d267-475c-8f76-bbe548843615 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:06.798054 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef886fa0-d267-475c-8f76-bbe548843615 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:06.817655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 519/1051] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:06.817655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef886fa0-d267-475c-8f76-bbe548843615 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:06.846982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a57dd-50a0-4a53-b6a1-b3e664ab4b44 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:06.867017 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:06.886007 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc30be-b29d-4d6b-99f7-756b3559aa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:06.886620 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc30be-b29d-4d6b-99f7-756b3559aa16 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:06.886898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc30be-b29d-4d6b-99f7-756b3559aa16 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:06.887176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc30be-b29d-4d6b-99f7-756b3559aa16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:06.890738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:06.931207 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef886fa0-d267-475c-8f76-bbe548843615 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:06.971650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc30be-b29d-4d6b-99f7-756b3559aa16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:06.973179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc30be-b29d-4d6b-99f7-756b3559aa16 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:06.974204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6c973-ce82-4493-b045-144f732edcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:06.975152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6c973-ce82-4493-b045-144f732edcf2 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:06.975461 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6c973-ce82-4493-b045-144f732edcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:06.975738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6c973-ce82-4493-b045-144f732edcf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:06.983804 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6c973-ce82-4493-b045-144f732edcf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:06.985302 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6c973-ce82-4493-b045-144f732edcf2 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:07.005099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-07e6c973-ce82-4493-b045-144f732edcf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:07.005099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:07.005099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:07.005099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:07.005099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:07.005099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:07.005099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:07.005099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:07.005099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:07.005099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:07.005099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:07.005099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:07.005099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:11:07.005099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:11:07.005099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:07.006136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:07.007022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:11:07.007022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:11:07.007022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:11:07.007022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:11:07.007022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:11:07.007022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:11:07.007022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:07.007022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:11:07.007461 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 520/1052] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:11:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:07.008825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6c973-ce82-4493-b045-144f732edcf2 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:07.105853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 533/1053] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:06 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:07.107826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc30be-b29d-4d6b-99f7-756b3559aa16 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:07.127433 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6c973-ce82-4493-b045-144f732edcf2 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:07.134946 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:07.148307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:07.166511 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1fede-5550-43ae-9d08-7e951d5c1aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:07.167140 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1fede-5550-43ae-9d08-7e951d5c1aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.167414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1fede-5550-43ae-9d08-7e951d5c1aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.167688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1fede-5550-43ae-9d08-7e951d5c1aa1 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.168325 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1fede-5550-43ae-9d08-7e951d5c1aa1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:07.168628 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1fede-5550-43ae-9d08-7e951d5c1aa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:07.170214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1fede-5550-43ae-9d08-7e951d5c1aa1 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:07.175274 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 521/1054] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:07.176075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1fede-5550-43ae-9d08-7e951d5c1aa1 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:07.224237 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc30be-b29d-4d6b-99f7-756b3559aa16 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:07.261791 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9fad1-4264-4b7e-b2b5-fa5b18ab285c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:07.262327 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9fad1-4264-4b7e-b2b5-fa5b18ab285c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.262604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9fad1-4264-4b7e-b2b5-fa5b18ab285c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.262925 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9fad1-4264-4b7e-b2b5-fa5b18ab285c None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.263600 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f9fad1-4264-4b7e-b2b5-fa5b18ab285c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:07.263897 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f9fad1-4264-4b7e-b2b5-fa5b18ab285c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:07.265351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f9fad1-4264-4b7e-b2b5-fa5b18ab285c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:07.277763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 534/1055] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:11:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:07.278989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9fad1-4264-4b7e-b2b5-fa5b18ab285c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:07.297932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1fede-5550-43ae-9d08-7e951d5c1aa1 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:07.340550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29348a4-8b56-4ee9-9d5d-69e6f1ac37e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:07.341792 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29348a4-8b56-4ee9-9d5d-69e6f1ac37e2 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.343025 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29348a4-8b56-4ee9-9d5d-69e6f1ac37e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.343025 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29348a4-8b56-4ee9-9d5d-69e6f1ac37e2 None admin] PATH_INFO: `/v3/projects/0b50f1329c4e4582ad7ed027748bd8e5` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.343971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29348a4-8b56-4ee9-9d5d-69e6f1ac37e2 None admin] RBAC: Authorizing `identity:get_project(project_id=0b50f1329c4e4582ad7ed027748bd8e5)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:07.345471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29348a4-8b56-4ee9-9d5d-69e6f1ac37e2 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:07.347164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 522/1056] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:11:07 2022] GET /identity/v3/projects/0b50f1329c4e4582ad7ed027748bd8e5 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:07.348006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29348a4-8b56-4ee9-9d5d-69e6f1ac37e2 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:07.392884 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:07.402807 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:07.406315 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9fad1-4264-4b7e-b2b5-fa5b18ab285c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:07.455281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc0365-1fb6-484d-9448-c5adf7208586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:07.456026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc0365-1fb6-484d-9448-c5adf7208586 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.456472 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc0365-1fb6-484d-9448-c5adf7208586 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.456884 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc0365-1fb6-484d-9448-c5adf7208586 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.457997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc0365-1fb6-484d-9448-c5adf7208586 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:07.459916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc0365-1fb6-484d-9448-c5adf7208586 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:07.473416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29348a4-8b56-4ee9-9d5d-69e6f1ac37e2 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:07.494717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 535/1057] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:11:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:07.495830 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc0365-1fb6-484d-9448-c5adf7208586 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:07.534718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af863f-4371-4d99-8f87-173c30e2de96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:07.535570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af863f-4371-4d99-8f87-173c30e2de96 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.535570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af863f-4371-4d99-8f87-173c30e2de96 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.535852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af863f-4371-4d99-8f87-173c30e2de96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.536569 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af863f-4371-4d99-8f87-173c30e2de96 None admin] RBAC: Adding query filter params (role.id=97c55e5f00404a29bb3f2df9566b6b0f, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=e3517b3b077a4a1fa76163657b46930c) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:07.536921 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af863f-4371-4d99-8f87-173c30e2de96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:07.538568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af863f-4371-4d99-8f87-173c30e2de96 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:07.548412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 523/1058] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:07 2022] GET /identity/v3/role_assignments?user.id=e3517b3b077a4a1fa76163657b46930c&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=97c55e5f00404a29bb3f2df9566b6b0f => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:07.655782 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc0365-1fb6-484d-9448-c5adf7208586 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:07.703495 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42218ef-9f9a-4ae8-b731-d2cc1cadb1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:07.704143 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42218ef-9f9a-4ae8-b731-d2cc1cadb1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.704428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42218ef-9f9a-4ae8-b731-d2cc1cadb1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.704705 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42218ef-9f9a-4ae8-b731-d2cc1cadb1d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.705544 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42218ef-9f9a-4ae8-b731-d2cc1cadb1d1 None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=0b50f1329c4e4582ad7ed027748bd8e5, user.id=40dc7e074b7946cab8a8812a655b8312) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:07.705838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42218ef-9f9a-4ae8-b731-d2cc1cadb1d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:07.707391 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42218ef-9f9a-4ae8-b731-d2cc1cadb1d1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:07.715355 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 536/1059] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:07 2022] GET /identity/v3/role_assignments?user.id=40dc7e074b7946cab8a8812a655b8312&scope.project.id=0b50f1329c4e4582ad7ed027748bd8e5&role.id=dc5d71ab685e420697f520aa3a368177 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:09.612811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5260f-c68e-48d1-82f2-7e52862f5195 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:09.613672 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cdca1-3239-41be-b194-c957ffdfadb7 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:09.614267 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5260f-c68e-48d1-82f2-7e52862f5195 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:09.614410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cdca1-3239-41be-b194-c957ffdfadb7 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:09.614543 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5260f-c68e-48d1-82f2-7e52862f5195 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:09.614682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cdca1-3239-41be-b194-c957ffdfadb7 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:09.615157 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 524/1060] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:09.616207 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 537/1061] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:09.620579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195cb8a-cafd-4091-b6b5-036f088250bf None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:09.624615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195cb8a-cafd-4091-b6b5-036f088250bf None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:09.624615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195cb8a-cafd-4091-b6b5-036f088250bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:09.624931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2e3b2-9e50-40fd-8c9c-025a1654337f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:09.624931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2e3b2-9e50-40fd-8c9c-025a1654337f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:09.624931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2e3b2-9e50-40fd-8c9c-025a1654337f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:09.646299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-c195cb8a-cafd-4091-b6b5-036f088250bf None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:09.647543 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c195cb8a-cafd-4091-b6b5-036f088250bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:09.648213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e2c2e3b2-9e50-40fd-8c9c-025a1654337f None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:09.649835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e2c2e3b2-9e50-40fd-8c9c-025a1654337f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:09.663394 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:09.755110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 538/1062] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:09.769087 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 525/1063] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:09.848797 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d2d6c-9d41-4729-ba00-edeb217ba086 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:09.849157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d2d6c-9d41-4729-ba00-edeb217ba086 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:09.849417 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d2d6c-9d41-4729-ba00-edeb217ba086 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:09.851041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 539/1064] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:09.856311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf72c97-f842-4b1b-80a4-cdad49c46a44 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:09.857203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf72c97-f842-4b1b-80a4-cdad49c46a44 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:09.857697 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf72c97-f842-4b1b-80a4-cdad49c46a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:09.877438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-acf72c97-f842-4b1b-80a4-cdad49c46a44 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:09.878568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-acf72c97-f842-4b1b-80a4-cdad49c46a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:09.905812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:09.958160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 526/1065] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:10.235792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d2d6c-9d41-4729-ba00-edeb217ba086 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:10.239987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf72c97-f842-4b1b-80a4-cdad49c46a44 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:10.351957 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d2d6c-9d41-4729-ba00-edeb217ba086 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:10.354893 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf72c97-f842-4b1b-80a4-cdad49c46a44 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:10.392729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a174bd-f3ce-4c62-b581-bb27b13ca35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:10.393510 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a174bd-f3ce-4c62-b581-bb27b13ca35d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:10.393792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a174bd-f3ce-4c62-b581-bb27b13ca35d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:10.394064 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a174bd-f3ce-4c62-b581-bb27b13ca35d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:10.398546 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad74a57-685e-4430-8419-42becf1d3d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:10.399163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad74a57-685e-4430-8419-42becf1d3d97 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:10.399446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad74a57-685e-4430-8419-42becf1d3d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:10.399799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad74a57-685e-4430-8419-42becf1d3d97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:10.408625 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a174bd-f3ce-4c62-b581-bb27b13ca35d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:10.413509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-67a174bd-f3ce-4c62-b581-bb27b13ca35d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:10.413509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:10.413509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:10.413509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:10.413509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:10.413509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:10.413509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:10.413509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:10.413509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:10.413509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:10.413509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:10.413509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:10.413509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:10.413509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:10.413509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:10.413509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:10.414842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:10.414842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:10.414842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:10.414842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:10.414842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:10.414842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:10.414842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:10.414842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:10.414842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:10.414842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:10.414842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:10.414842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:10.414842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:10.414842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:10.414842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:10.414842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:10.416519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:10.416519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:10.416519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:10.416519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:10.416519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:10.416519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:10.416519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:10.416519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:10.416519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:10.416519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:10.416519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:10.416519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:10.416519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 540/1066] 10.222.0.88 () {56 vars in 1120 bytes} [Tue Aug 23 22:11:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:10.416519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a174bd-f3ce-4c62-b581-bb27b13ca35d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:10.422983 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:10.441307 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:10.489876 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad74a57-685e-4430-8419-42becf1d3d97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:10.491287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad74a57-685e-4430-8419-42becf1d3d97 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:10.539273 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a174bd-f3ce-4c62-b581-bb27b13ca35d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:10.579748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b268a3d-a05f-4ad3-aebd-9df007082594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:10.580386 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b268a3d-a05f-4ad3-aebd-9df007082594 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:10.580661 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b268a3d-a05f-4ad3-aebd-9df007082594 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:10.580937 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b268a3d-a05f-4ad3-aebd-9df007082594 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:10.592385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b268a3d-a05f-4ad3-aebd-9df007082594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:10.597227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-5b268a3d-a05f-4ad3-aebd-9df007082594 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:10.597227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:10.597227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:10.597227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:10.597227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:10.597227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:10.597227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:10.597227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:10.597227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:10.597227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:10.597227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:10.597227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:10.597227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:10.597227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:10.597227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:10.597227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:10.598784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:10.598784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:10.598784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:10.598784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:10.598784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:10.598784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:10.598784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:10.598784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:10.598784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:10.598784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:10.598784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:10.598784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:10.598784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:10.598784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:10.598784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:10.598784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:10.600442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:10.600442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:10.600442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:10.600442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:10.600442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:10.600442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:10.600442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:10.600442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:10.600442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:10.600442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:10.600442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:10.600442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:10.600442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 541/1067] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:11:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:10.600442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b268a3d-a05f-4ad3-aebd-9df007082594 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:10.608175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 527/1068] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 22:11:10.608175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad74a57-685e-4430-8419-42becf1d3d97 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:10.682275 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:10.688452 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:10.728682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b268a3d-a05f-4ad3-aebd-9df007082594 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:10.728923 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad74a57-685e-4430-8419-42becf1d3d97 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:10.770967 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85752e3e-aa5d-4561-b659-461d87f31dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:10.771633 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85752e3e-aa5d-4561-b659-461d87f31dda None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:10.771991 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85752e3e-aa5d-4561-b659-461d87f31dda None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:10.772343 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85752e3e-aa5d-4561-b659-461d87f31dda None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:10.773218 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85752e3e-aa5d-4561-b659-461d87f31dda None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:10.773633 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85752e3e-aa5d-4561-b659-461d87f31dda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:10.775091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85752e3e-aa5d-4561-b659-461d87f31dda None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:10.777457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7c89a-57f9-438d-ac76-56f7e2e46a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:10.778210 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7c89a-57f9-438d-ac76-56f7e2e46a15 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:10.779174 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7c89a-57f9-438d-ac76-56f7e2e46a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:10.779498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7c89a-57f9-438d-ac76-56f7e2e46a15 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:10.780496 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7c89a-57f9-438d-ac76-56f7e2e46a15 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:10.780860 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7c89a-57f9-438d-ac76-56f7e2e46a15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:10.782389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7c89a-57f9-438d-ac76-56f7e2e46a15 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:10.784548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 528/1069] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:11:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:10.785412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85752e3e-aa5d-4561-b659-461d87f31dda None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:10.792590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 542/1070] 10.222.0.88 () {56 vars in 1115 bytes} [Tue Aug 23 22:11:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:10.793778 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7c89a-57f9-438d-ac76-56f7e2e46a15 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:10.904311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85752e3e-aa5d-4561-b659-461d87f31dda None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:10.919902 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7c89a-57f9-438d-ac76-56f7e2e46a15 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:10.926444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:10.943542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3048eff-f0d9-4b9c-bc2f-ab5629c16922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:10.944273 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3048eff-f0d9-4b9c-bc2f-ab5629c16922 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:10.944556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3048eff-f0d9-4b9c-bc2f-ab5629c16922 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:10.944837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3048eff-f0d9-4b9c-bc2f-ab5629c16922 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:10.948988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:10.958723 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3048eff-f0d9-4b9c-bc2f-ab5629c16922 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:10.960382 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3048eff-f0d9-4b9c-bc2f-ab5629c16922 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:10.961292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cba6e-a409-4dac-b04c-fa99ced6f527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:10.961990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cba6e-a409-4dac-b04c-fa99ced6f527 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:10.962355 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cba6e-a409-4dac-b04c-fa99ced6f527 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:10.962688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cba6e-a409-4dac-b04c-fa99ced6f527 None admin] PATH_INFO: `/v3/users/40dc7e074b7946cab8a8812a655b8312` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:10.964076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3cba6e-a409-4dac-b04c-fa99ced6f527 None admin] RBAC: Authorizing `identity:get_user(user_id=40dc7e074b7946cab8a8812a655b8312)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:10.966387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3cba6e-a409-4dac-b04c-fa99ced6f527 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:10.968163 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 543/1071] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:10 2022] GET /identity/v3/users/40dc7e074b7946cab8a8812a655b8312 => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:10.968957 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cba6e-a409-4dac-b04c-fa99ced6f527 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:10.980492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-b3048eff-f0d9-4b9c-bc2f-ab5629c16922 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:10.980492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:10.980492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:10.980492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:10.980492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:10.980492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:10.980492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:10.980492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:10.980492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:10.980492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:10.980492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:10.980492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:10.980492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:10.980492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:10.980492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:10.980492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:10.989762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:10.990744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:10.990744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:10.990744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:11:10.990744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 529/1072] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:10.990744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3048eff-f0d9-4b9c-bc2f-ab5629c16922 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:11.099750 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cba6e-a409-4dac-b04c-fa99ced6f527 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:11.126598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3048eff-f0d9-4b9c-bc2f-ab5629c16922 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:11.150682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3210f4-98b0-4096-81f8-061031916ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:11.151313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3210f4-98b0-4096-81f8-061031916ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:11.151588 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3210f4-98b0-4096-81f8-061031916ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:11.151865 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3210f4-98b0-4096-81f8-061031916ae6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:11.164721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3210f4-98b0-4096-81f8-061031916ae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:11.166687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3210f4-98b0-4096-81f8-061031916ae6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:11.167891 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5f0e7-5cd1-4a0d-aca9-ec1637a79f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:11.168541 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5f0e7-5cd1-4a0d-aca9-ec1637a79f3e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:11.169360 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5f0e7-5cd1-4a0d-aca9-ec1637a79f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:11.169645 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5f0e7-5cd1-4a0d-aca9-ec1637a79f3e None admin] PATH_INFO: `/v3/projects/0b50f1329c4e4582ad7ed027748bd8e5` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:11.170873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5f0e7-5cd1-4a0d-aca9-ec1637a79f3e None admin] RBAC: Authorizing `identity:get_project(project_id=0b50f1329c4e4582ad7ed027748bd8e5)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:11.172281 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5f0e7-5cd1-4a0d-aca9-ec1637a79f3e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:11.173995 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 530/1073] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:11:10 2022] GET /identity/v3/projects/0b50f1329c4e4582ad7ed027748bd8e5 => generated 311 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:11.174767 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5f0e7-5cd1-4a0d-aca9-ec1637a79f3e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:11.191421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-3c3210f4-98b0-4096-81f8-061031916ae6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:11.191421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:11.191421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:11.191421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:11.191421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:11.191421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:11.191421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:11.191421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:11.191421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:11.191421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:11.191421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:11.191421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:11.191421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:11.191421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:11.191421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:11.191421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:11.192464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:11.193434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:11.193434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:11.193434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:11.193434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 544/1074] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:11.194159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3210f4-98b0-4096-81f8-061031916ae6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:11.240525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:11.247451 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:11.311369 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5f0e7-5cd1-4a0d-aca9-ec1637a79f3e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:11.314791 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3210f4-98b0-4096-81f8-061031916ae6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:11.359234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f113085-0ea4-4052-9ffc-aa21b4673f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:11.359820 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f113085-0ea4-4052-9ffc-aa21b4673f75 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:11.360104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f113085-0ea4-4052-9ffc-aa21b4673f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:11.360381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f113085-0ea4-4052-9ffc-aa21b4673f75 None admin] PATH_INFO: `/v3/projects/0b50f1329c4e4582ad7ed027748bd8e5/users/40dc7e074b7946cab8a8812a655b8312/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:11.360511 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9ca0b-2472-44f8-b6f8-375153df2df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:11.361081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9ca0b-2472-44f8-b6f8-375153df2df1 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:11.361362 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9ca0b-2472-44f8-b6f8-375153df2df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:11.361638 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9ca0b-2472-44f8-b6f8-375153df2df1 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:11.362293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9ca0b-2472-44f8-b6f8-375153df2df1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:11.362447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f113085-0ea4-4052-9ffc-aa21b4673f75 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b50f1329c4e4582ad7ed027748bd8e5, user_id=40dc7e074b7946cab8a8812a655b8312, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:11.362586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9ca0b-2472-44f8-b6f8-375153df2df1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:11.363902 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f113085-0ea4-4052-9ffc-aa21b4673f75 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:11.364051 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9ca0b-2472-44f8-b6f8-375153df2df1 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:11.383926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 531/1075] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:11.385334 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9ca0b-2472-44f8-b6f8-375153df2df1 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:11.386506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 545/1076] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:11:11 2022] PUT /identity/v3/projects/0b50f1329c4e4582ad7ed027748bd8e5/users/40dc7e074b7946cab8a8812a655b8312/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:11:11.598496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:11.602224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9ca0b-2472-44f8-b6f8-375153df2df1 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:11.657381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7a581-83a4-4a85-8775-415e0791eab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:11.658492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7a581-83a4-4a85-8775-415e0791eab8 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:11.658993 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7a581-83a4-4a85-8775-415e0791eab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:11.659700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7a581-83a4-4a85-8775-415e0791eab8 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:11.660951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7a581-83a4-4a85-8775-415e0791eab8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:11.661532 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7a581-83a4-4a85-8775-415e0791eab8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:11.663784 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7a581-83a4-4a85-8775-415e0791eab8 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:11.686840 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 532/1077] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:11.696596 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f113085-0ea4-4052-9ffc-aa21b4673f75 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:11.858973 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f113085-0ea4-4052-9ffc-aa21b4673f75 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:11.868315 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:11.907934 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054574f-facb-44ff-9e5c-6273f715a88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:11.908938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054574f-facb-44ff-9e5c-6273f715a88f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:11.909501 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054574f-facb-44ff-9e5c-6273f715a88f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:11.909951 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054574f-facb-44ff-9e5c-6273f715a88f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:11.923358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054574f-facb-44ff-9e5c-6273f715a88f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:11.926089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054574f-facb-44ff-9e5c-6273f715a88f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:11.951277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-d054574f-facb-44ff-9e5c-6273f715a88f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:11:11.951277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:11.951277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:11.951277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:11.951277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:11.951277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:11.951277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:11.951277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:11.951277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:11.951277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:11.951277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:11.951277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:11.951277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:11:11.951277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:11:11.951277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:11:11.951277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:11.952965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:11.954732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:11.954732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:11.954732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:11:11.954732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:11:11.954732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:11:11.954732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:11:11.954732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:11:11.954732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:11:11.954732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:11:11.954732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:11.954732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 546/1078] 10.222.0.88 () {56 vars in 1093 bytes} [Tue Aug 23 22:11:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:11.961919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7a581-83a4-4a85-8775-415e0791eab8 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:12.098858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7a581-83a4-4a85-8775-415e0791eab8 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:12.149392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a05c57-79f7-4d35-a86d-5f85c22f72ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:12.150054 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a05c57-79f7-4d35-a86d-5f85c22f72ab None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:12.150512 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a05c57-79f7-4d35-a86d-5f85c22f72ab None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:12.150881 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a05c57-79f7-4d35-a86d-5f85c22f72ab None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:12.152026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a05c57-79f7-4d35-a86d-5f85c22f72ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:12.152350 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a05c57-79f7-4d35-a86d-5f85c22f72ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:12.155071 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a05c57-79f7-4d35-a86d-5f85c22f72ab None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:12.190074 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 533/1079] 10.222.0.88 () {56 vars in 1133 bytes} [Tue Aug 23 22:11:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:12.196578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054574f-facb-44ff-9e5c-6273f715a88f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:12.343101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054574f-facb-44ff-9e5c-6273f715a88f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:12.386519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf78fd9-b592-46ae-a7f3-2ad1365a38e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:12.387197 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf78fd9-b592-46ae-a7f3-2ad1365a38e3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:12.388026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf78fd9-b592-46ae-a7f3-2ad1365a38e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:12.388341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf78fd9-b592-46ae-a7f3-2ad1365a38e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:12.405797 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf78fd9-b592-46ae-a7f3-2ad1365a38e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:12.407359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf78fd9-b592-46ae-a7f3-2ad1365a38e3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:12.424298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-adf78fd9-b592-46ae-a7f3-2ad1365a38e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:12.424298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:12.424298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:12.424298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:12.424298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:12.424298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:12.424298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:12.424298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:12.424298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:12.424298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:12.424298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:12.424298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:12.424298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:12.424298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:12.424298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:12.424298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:12.425498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:12.427131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:12.427131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:12.427131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:12.427131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 547/1080] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:12.432815 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a05c57-79f7-4d35-a86d-5f85c22f72ab None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:12.457959 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:12.579660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a05c57-79f7-4d35-a86d-5f85c22f72ab None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:12.627916 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a44ae1-55f7-447a-8a05-ee49acd20409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:12.628652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a44ae1-55f7-447a-8a05-ee49acd20409 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:12.628974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a44ae1-55f7-447a-8a05-ee49acd20409 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:12.629280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a44ae1-55f7-447a-8a05-ee49acd20409 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:12.630050 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a44ae1-55f7-447a-8a05-ee49acd20409 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:12.630384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a44ae1-55f7-447a-8a05-ee49acd20409 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:12.631832 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a44ae1-55f7-447a-8a05-ee49acd20409 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:12.653854 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 534/1081] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:12.655453 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf78fd9-b592-46ae-a7f3-2ad1365a38e3 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:12.688572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:12.819427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf78fd9-b592-46ae-a7f3-2ad1365a38e3 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:12.868884 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147584f3-a500-402c-adfe-b91e98f95e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:12.869827 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147584f3-a500-402c-adfe-b91e98f95e06 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:12.870167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147584f3-a500-402c-adfe-b91e98f95e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:12.870505 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147584f3-a500-402c-adfe-b91e98f95e06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:12.881585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147584f3-a500-402c-adfe-b91e98f95e06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:12.883875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147584f3-a500-402c-adfe-b91e98f95e06 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:12.903711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-147584f3-a500-402c-adfe-b91e98f95e06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:12.903711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:12.903711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:12.903711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:12.903711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:12.903711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:12.903711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:12.903711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:12.903711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:12.903711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:12.903711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:12.903711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:12.903711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:11:12.903711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:11:12.903711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:12.905331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:12.906285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:11:12.906285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:11:12.906285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:11:12.906285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:11:12.906285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:11:12.906285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:11:12.906285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:12.906285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:12.906285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 548/1082] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:11:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:12.913192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a44ae1-55f7-447a-8a05-ee49acd20409 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:13.015498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:13.048093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a44ae1-55f7-447a-8a05-ee49acd20409 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:13.102005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef86ad-c95d-4ff0-a929-09437e452544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:13.102612 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef86ad-c95d-4ff0-a929-09437e452544 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:13.102921 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef86ad-c95d-4ff0-a929-09437e452544 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:13.103354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef86ad-c95d-4ff0-a929-09437e452544 None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:13.104012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef86ad-c95d-4ff0-a929-09437e452544 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:13.104320 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef86ad-c95d-4ff0-a929-09437e452544 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:13.105867 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef86ad-c95d-4ff0-a929-09437e452544 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:13.121466 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 535/1083] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:11:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:13.127763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147584f3-a500-402c-adfe-b91e98f95e06 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:13.241571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:13.267354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147584f3-a500-402c-adfe-b91e98f95e06 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:13.308715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee3927-ae6f-4bb8-99be-0b910776e82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:13.310074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee3927-ae6f-4bb8-99be-0b910776e82b None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:13.311015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee3927-ae6f-4bb8-99be-0b910776e82b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:13.311015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee3927-ae6f-4bb8-99be-0b910776e82b None admin] PATH_INFO: `/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/e3517b3b077a4a1fa76163657b46930c/roles/97c55e5f00404a29bb3f2df9566b6b0f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:13.313379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee3927-ae6f-4bb8-99be-0b910776e82b None admin] RBAC: Authorizing `identity:create_grant(project_id=8a7317af325f485e8fae49631cb5bc20, user_id=e3517b3b077a4a1fa76163657b46930c, role_id=97c55e5f00404a29bb3f2df9566b6b0f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:13.314902 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee3927-ae6f-4bb8-99be-0b910776e82b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:13.330982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 549/1084] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:11:13 2022] PUT /identity/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/e3517b3b077a4a1fa76163657b46930c/roles/97c55e5f00404a29bb3f2df9566b6b0f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:11:13.563221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79780c4-7dcc-47d8-9138-b845a372703a None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:13.563738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79780c4-7dcc-47d8-9138-b845a372703a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:13.564239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79780c4-7dcc-47d8-9138-b845a372703a None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:13.566936 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 536/1085] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:13.577412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56900db0-3694-4a6e-bf10-56c909731816 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:13.577929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56900db0-3694-4a6e-bf10-56c909731816 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:13.578434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56900db0-3694-4a6e-bf10-56c909731816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:13.606261 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-56900db0-3694-4a6e-bf10-56c909731816 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:13.608465 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-56900db0-3694-4a6e-bf10-56c909731816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:13.717868 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 550/1086] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:13.991525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79780c4-7dcc-47d8-9138-b845a372703a None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:14.113976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79780c4-7dcc-47d8-9138-b845a372703a None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:14.155597 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980ef1b-c0ed-491e-a1ab-1d0afee5d436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:14.156232 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980ef1b-c0ed-491e-a1ab-1d0afee5d436 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:14.156509 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980ef1b-c0ed-491e-a1ab-1d0afee5d436 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:14.156775 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980ef1b-c0ed-491e-a1ab-1d0afee5d436 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:14.165145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:14.173314 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980ef1b-c0ed-491e-a1ab-1d0afee5d436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:14.179689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-e980ef1b-c0ed-491e-a1ab-1d0afee5d436 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:14.179689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:14.179689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:14.179689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:14.179689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:14.179689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:14.179689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:14.179689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:14.179689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:14.179689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:14.179689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:14.179689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:14.179689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:14.179689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:14.179689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:14.179689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:14.181536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:14.181536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:14.181536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:14.181536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:14.181536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:14.181536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:14.181536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:14.181536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:14.181536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:14.181536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:14.181536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:14.181536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:14.181536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:14.181536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:14.181536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:14.181536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:14.183151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:14.183151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:14.183151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:14.183151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:14.183151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:14.183151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:14.183151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:14.183151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:14.183151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:14.183151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:14.183151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:14.183151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:11:14.183151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 537/1087] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:11:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:14.189253 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56900db0-3694-4a6e-bf10-56c909731816 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:14.260123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:14.299627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56900db0-3694-4a6e-bf10-56c909731816 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:14.337290 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a880d55-bbd6-4cca-90fb-75fefad5a939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:14.337904 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a880d55-bbd6-4cca-90fb-75fefad5a939 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:14.338182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a880d55-bbd6-4cca-90fb-75fefad5a939 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:14.338452 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a880d55-bbd6-4cca-90fb-75fefad5a939 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:14.339124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a880d55-bbd6-4cca-90fb-75fefad5a939 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:14.339471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a880d55-bbd6-4cca-90fb-75fefad5a939 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:14.340966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a880d55-bbd6-4cca-90fb-75fefad5a939 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:14.350155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 551/1088] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:11:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:14.356270 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980ef1b-c0ed-491e-a1ab-1d0afee5d436 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:14.475856 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980ef1b-c0ed-491e-a1ab-1d0afee5d436 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:14.520717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f164aa7-631c-4cc7-a697-722bc77182a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:14.521430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f164aa7-631c-4cc7-a697-722bc77182a8 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:14.521781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f164aa7-631c-4cc7-a697-722bc77182a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:14.522098 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f164aa7-631c-4cc7-a697-722bc77182a8 None admin] PATH_INFO: `/v3/users/40dc7e074b7946cab8a8812a655b8312` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:14.523579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f164aa7-631c-4cc7-a697-722bc77182a8 None admin] RBAC: Authorizing `identity:get_user(user_id=40dc7e074b7946cab8a8812a655b8312)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:14.525312 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f164aa7-631c-4cc7-a697-722bc77182a8 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:14.526927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 538/1089] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:14 2022] GET /identity/v3/users/40dc7e074b7946cab8a8812a655b8312 => generated 285 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:14.533997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a880d55-bbd6-4cca-90fb-75fefad5a939 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:14.640946 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a880d55-bbd6-4cca-90fb-75fefad5a939 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:14.671153 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:14.677964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdef44-1811-4fb8-809b-1428bfb83054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:14.678582 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdef44-1811-4fb8-809b-1428bfb83054 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:14.678861 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdef44-1811-4fb8-809b-1428bfb83054 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:14.679135 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdef44-1811-4fb8-809b-1428bfb83054 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:14.752191 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafdef44-1811-4fb8-809b-1428bfb83054 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:14.753683 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafdef44-1811-4fb8-809b-1428bfb83054 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:14.867976 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 552/1090] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 22:11:14.874036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f164aa7-631c-4cc7-a697-722bc77182a8 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:14.944934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:14.982776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f164aa7-631c-4cc7-a697-722bc77182a8 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:15.019802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2417e-6efa-48bb-8141-ac60dd7435dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:15.020489 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2417e-6efa-48bb-8141-ac60dd7435dd None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:15.020704 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2417e-6efa-48bb-8141-ac60dd7435dd None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:15.020973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2417e-6efa-48bb-8141-ac60dd7435dd None admin] PATH_INFO: `/v3/projects/0b50f1329c4e4582ad7ed027748bd8e5` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:15.022129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2417e-6efa-48bb-8141-ac60dd7435dd None admin] RBAC: Authorizing `identity:get_project(project_id=0b50f1329c4e4582ad7ed027748bd8e5)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:15.023607 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2417e-6efa-48bb-8141-ac60dd7435dd None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:15.025269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 539/1091] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:11:14 2022] GET /identity/v3/projects/0b50f1329c4e4582ad7ed027748bd8e5 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:15.033308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdef44-1811-4fb8-809b-1428bfb83054 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:15.071800 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:15.182091 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdef44-1811-4fb8-809b-1428bfb83054 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:15.221944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c81cae-ac49-4a36-9c7d-0d3cb8aaaa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:15.222683 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c81cae-ac49-4a36-9c7d-0d3cb8aaaa12 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:15.222834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c81cae-ac49-4a36-9c7d-0d3cb8aaaa12 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:15.223082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c81cae-ac49-4a36-9c7d-0d3cb8aaaa12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:15.223807 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c81cae-ac49-4a36-9c7d-0d3cb8aaaa12 None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=0b50f1329c4e4582ad7ed027748bd8e5, user.id=40dc7e074b7946cab8a8812a655b8312) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:15.224115 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c81cae-ac49-4a36-9c7d-0d3cb8aaaa12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:15.225585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c81cae-ac49-4a36-9c7d-0d3cb8aaaa12 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:15.232425 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 553/1092] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:15 2022] GET /identity/v3/role_assignments?user.id=40dc7e074b7946cab8a8812a655b8312&scope.project.id=0b50f1329c4e4582ad7ed027748bd8e5&role.id=dc5d71ab685e420697f520aa3a368177 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:15.446918 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2178b-cbf0-466f-915e-1221172296aa None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:15.447182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2178b-cbf0-466f-915e-1221172296aa None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:15.447448 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2178b-cbf0-466f-915e-1221172296aa None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:15.448813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 540/1093] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:15.455333 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc13f9-6775-497d-a820-6722d69c21f2 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:15.455616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc13f9-6775-497d-a820-6722d69c21f2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:15.455894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc13f9-6775-497d-a820-6722d69c21f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:15.484102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-babc13f9-6775-497d-a820-6722d69c21f2 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:15.485494 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-babc13f9-6775-497d-a820-6722d69c21f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:15.562601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 554/1094] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:15.838408 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2178b-cbf0-466f-915e-1221172296aa None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:15.972094 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2178b-cbf0-466f-915e-1221172296aa None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:16.027000 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dc746-5d15-47c9-871e-587dc4d7b7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:16.028877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dc746-5d15-47c9-871e-587dc4d7b7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:16.028877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dc746-5d15-47c9-871e-587dc4d7b7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:16.028877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dc746-5d15-47c9-871e-587dc4d7b7c2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:16.032602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:16.045927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562dc746-5d15-47c9-871e-587dc4d7b7c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:16.051421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-562dc746-5d15-47c9-871e-587dc4d7b7c2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:16.051421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:16.051421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:16.051421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:16.051421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:16.051421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:16.051421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:16.051421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:16.051421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:16.051421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:16.051421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:16.051421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:16.051421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:16.051421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:16.051421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:16.051421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:16.054515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:16.054515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:16.054515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:16.054515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:16.054515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:16.054515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:16.054515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:16.054515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:16.054515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:16.054515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:16.054515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:16.054515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:16.054515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:16.054515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:16.054515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:16.054515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:16.057938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:16.057938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:16.057938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:16.057938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:16.057938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:16.057938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:16.057938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:16.057938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:16.057938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:16.057938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:16.057938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:16.057938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:11:16.057938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 541/1095] 10.222.0.88 () {56 vars in 1120 bytes} [Tue Aug 23 22:11:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:16.061555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc13f9-6775-497d-a820-6722d69c21f2 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:16.091168 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:16.187365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc13f9-6775-497d-a820-6722d69c21f2 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:16.238714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603d7f8-9e20-4301-b2d7-1b241acf9c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:16.239494 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603d7f8-9e20-4301-b2d7-1b241acf9c8f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:16.239629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603d7f8-9e20-4301-b2d7-1b241acf9c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:16.239869 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603d7f8-9e20-4301-b2d7-1b241acf9c8f None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:16.240548 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603d7f8-9e20-4301-b2d7-1b241acf9c8f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:16.240857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603d7f8-9e20-4301-b2d7-1b241acf9c8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:16.242295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603d7f8-9e20-4301-b2d7-1b241acf9c8f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:16.253961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 555/1096] 10.222.0.88 () {56 vars in 1115 bytes} [Tue Aug 23 22:11:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:16.262262 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dc746-5d15-47c9-871e-587dc4d7b7c2 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:16.381871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dc746-5d15-47c9-871e-587dc4d7b7c2 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:16.434031 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239978db-e4f3-49e3-b5c8-58a96458f5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:16.436139 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239978db-e4f3-49e3-b5c8-58a96458f5bf None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:16.436556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239978db-e4f3-49e3-b5c8-58a96458f5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:16.437012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239978db-e4f3-49e3-b5c8-58a96458f5bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:16.453726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239978db-e4f3-49e3-b5c8-58a96458f5bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:16.456422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239978db-e4f3-49e3-b5c8-58a96458f5bf None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:16.483164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-239978db-e4f3-49e3-b5c8-58a96458f5bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:16.483164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:16.483164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:16.483164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:16.483164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:16.483164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:16.483164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:16.483164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:16.483164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:16.483164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:16.483164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:16.483164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:16.483164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:16.483164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:16.483164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:16.483164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:16.484736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:16.486830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:16.486830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:16.486830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:11:16.486830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 542/1097] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:16.495053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603d7f8-9e20-4301-b2d7-1b241acf9c8f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:16.591919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:16.608277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603d7f8-9e20-4301-b2d7-1b241acf9c8f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:16.645473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870eeac-a9ca-4afe-8c10-89d0120da1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:16.645989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870eeac-a9ca-4afe-8c10-89d0120da1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:16.646352 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870eeac-a9ca-4afe-8c10-89d0120da1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:16.646671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870eeac-a9ca-4afe-8c10-89d0120da1f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:16.647319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870eeac-a9ca-4afe-8c10-89d0120da1f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:16.647628 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870eeac-a9ca-4afe-8c10-89d0120da1f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:16.649036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870eeac-a9ca-4afe-8c10-89d0120da1f1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:16.661961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 556/1098] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:16.667708 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239978db-e4f3-49e3-b5c8-58a96458f5bf None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:16.719849 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:16.787806 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239978db-e4f3-49e3-b5c8-58a96458f5bf None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:16.825193 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6be38-3203-4864-bb8c-897be3dab6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:16.825837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6be38-3203-4864-bb8c-897be3dab6ec None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:16.826265 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6be38-3203-4864-bb8c-897be3dab6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:16.826544 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6be38-3203-4864-bb8c-897be3dab6ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:16.834895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6be38-3203-4864-bb8c-897be3dab6ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:16.836346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6be38-3203-4864-bb8c-897be3dab6ec None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:16.849196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-d9c6be38-3203-4864-bb8c-897be3dab6ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:11:16.849196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:16.849196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:16.849196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:16.849196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:16.849196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:16.849196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:16.849196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:16.849196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:16.849196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:16.849196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:16.849196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:16.849196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:11:16.849196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:11:16.849196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:11:16.849196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:16.851036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:16.852726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:16.852726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:16.852726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:11:16.852726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:11:16.852726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:11:16.852726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:11:16.852726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:11:16.852726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:11:16.852726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:11:16.852726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:11:16.852726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 543/1099] 10.222.0.88 () {56 vars in 1093 bytes} [Tue Aug 23 22:11:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:16.856799 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870eeac-a9ca-4afe-8c10-89d0120da1f1 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:16.973871 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870eeac-a9ca-4afe-8c10-89d0120da1f1 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:17.013533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd938bc-3be6-4278-9fa5-3f55476fca21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:17.014239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd938bc-3be6-4278-9fa5-3f55476fca21 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.014428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd938bc-3be6-4278-9fa5-3f55476fca21 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.014698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd938bc-3be6-4278-9fa5-3f55476fca21 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.015434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd938bc-3be6-4278-9fa5-3f55476fca21 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:17.015742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd938bc-3be6-4278-9fa5-3f55476fca21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:17.017199 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd938bc-3be6-4278-9fa5-3f55476fca21 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:17.041530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 557/1100] 10.222.0.88 () {56 vars in 1133 bytes} [Tue Aug 23 22:11:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:17.042778 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6be38-3203-4864-bb8c-897be3dab6ec None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:17.148916 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6be38-3203-4864-bb8c-897be3dab6ec None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:17.156671 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:17.189123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b20e3-ee9a-4dce-b189-ea4d9d94e8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:17.189746 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b20e3-ee9a-4dce-b189-ea4d9d94e8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.190023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b20e3-ee9a-4dce-b189-ea4d9d94e8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.190293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b20e3-ee9a-4dce-b189-ea4d9d94e8e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.281471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b20e3-ee9a-4dce-b189-ea4d9d94e8e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:17.282848 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b20e3-ee9a-4dce-b189-ea4d9d94e8e3 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:17.359898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b22fe-7850-4c19-af60-a841807d3c56 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.360468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b22fe-7850-4c19-af60-a841807d3c56 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.360468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b22fe-7850-4c19-af60-a841807d3c56 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.361893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 558/1101] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:11:17.367591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b2329-336d-42d2-b91d-883d3a9f62ba None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.367911 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b2329-336d-42d2-b91d-883d3a9f62ba None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.368193 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b2329-336d-42d2-b91d-883d3a9f62ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.375665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:17.388424 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 544/1102] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:17.392034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2d0b2329-336d-42d2-b91d-883d3a9f62ba None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:17.393529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2d0b2329-336d-42d2-b91d-883d3a9f62ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:17.396574 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b20e3-ee9a-4dce-b189-ea4d9d94e8e3 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:17.406506 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:17.518335 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 559/1103] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:17.530634 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b20e3-ee9a-4dce-b189-ea4d9d94e8e3 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:17.568871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbe554-3a99-4f7a-b58c-afd8b4f96a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:17.569455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbe554-3a99-4f7a-b58c-afd8b4f96a79 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.569736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbe554-3a99-4f7a-b58c-afd8b4f96a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.570008 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbe554-3a99-4f7a-b58c-afd8b4f96a79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.581486 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbe554-3a99-4f7a-b58c-afd8b4f96a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:17.582971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbe554-3a99-4f7a-b58c-afd8b4f96a79 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:17.597754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-e2fbe554-3a99-4f7a-b58c-afd8b4f96a79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:17.597754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:17.597754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:17.597754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:17.597754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:17.597754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:17.597754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:17.597754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:17.597754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:17.597754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:17.597754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:17.597754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:17.597754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:17.597754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:17.597754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:17.597754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:17.598738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:17.599651 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:17.599651 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:17.599651 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:11:17.599651 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 545/1104] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:17.604716 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b2329-336d-42d2-b91d-883d3a9f62ba None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:17.709029 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b2329-336d-42d2-b91d-883d3a9f62ba None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:17.745569 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8835a-6462-4a50-94d5-973339f623d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:17.746176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8835a-6462-4a50-94d5-973339f623d6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.746452 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8835a-6462-4a50-94d5-973339f623d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.747597 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8835a-6462-4a50-94d5-973339f623d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.748260 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8835a-6462-4a50-94d5-973339f623d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:17.748559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8835a-6462-4a50-94d5-973339f623d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:17.749976 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8835a-6462-4a50-94d5-973339f623d6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:17.751741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:17.766888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 560/1105] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:17.773448 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbe554-3a99-4f7a-b58c-afd8b4f96a79 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:17.813794 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8835a-6462-4a50-94d5-973339f623d6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:17.862727 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:17.915860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbe554-3a99-4f7a-b58c-afd8b4f96a79 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:17.957856 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803260c-63f3-45ac-8e1a-8e7fff34cf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:17.958609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803260c-63f3-45ac-8e1a-8e7fff34cf19 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.958890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803260c-63f3-45ac-8e1a-8e7fff34cf19 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.959163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803260c-63f3-45ac-8e1a-8e7fff34cf19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.962182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8835a-6462-4a50-94d5-973339f623d6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:17.972040 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803260c-63f3-45ac-8e1a-8e7fff34cf19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:17.974566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803260c-63f3-45ac-8e1a-8e7fff34cf19 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:18.003637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-d803260c-63f3-45ac-8e1a-8e7fff34cf19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:18.003637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:18.003637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:18.003637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:18.003637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:18.003637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:18.003637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:18.003637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:18.003637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:18.003637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:18.003637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:18.003637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:18.003637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:11:18.003637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:11:18.003637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:18.004660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:18.005926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:11:18.005926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:11:18.005926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:11:18.005926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:11:18.005926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:11:18.005926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:11:18.005926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:18.005926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:11:18.006367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3f894-be6b-43f9-a2f9-673cb46e98d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:18.006473 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 546/1106] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:11:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:18.006608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3f894-be6b-43f9-a2f9-673cb46e98d8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:18.006608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3f894-be6b-43f9-a2f9-673cb46e98d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:18.006608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3f894-be6b-43f9-a2f9-673cb46e98d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:18.007533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3f894-be6b-43f9-a2f9-673cb46e98d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:18.009006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3f894-be6b-43f9-a2f9-673cb46e98d8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:18.011492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 561/1107] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:18.013297 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803260c-63f3-45ac-8e1a-8e7fff34cf19 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:18.018217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3f894-be6b-43f9-a2f9-673cb46e98d8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:18.095400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:18.144219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803260c-63f3-45ac-8e1a-8e7fff34cf19 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:18.155541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3f894-be6b-43f9-a2f9-673cb46e98d8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:18.164382 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:18.186065 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4a959-16c7-4075-bb45-8de0f8111f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:18.186690 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4a959-16c7-4075-bb45-8de0f8111f50 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:18.186975 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4a959-16c7-4075-bb45-8de0f8111f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:18.187247 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4a959-16c7-4075-bb45-8de0f8111f50 None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:18.187950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4a959-16c7-4075-bb45-8de0f8111f50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:18.188259 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4a959-16c7-4075-bb45-8de0f8111f50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:18.189741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4a959-16c7-4075-bb45-8de0f8111f50 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:18.196609 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e844000-ceca-4cba-9532-5f8f5f1112d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:18.197259 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e844000-ceca-4cba-9532-5f8f5f1112d3 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:18.197541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e844000-ceca-4cba-9532-5f8f5f1112d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:18.197811 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e844000-ceca-4cba-9532-5f8f5f1112d3 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:18.198457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e844000-ceca-4cba-9532-5f8f5f1112d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:18.199946 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e844000-ceca-4cba-9532-5f8f5f1112d3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:18.203602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 547/1108] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:11:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:18.213736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4a959-16c7-4075-bb45-8de0f8111f50 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:18.225402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 562/1109] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:11:18 2022] POST /identity/v3/users => generated 286 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:11:18.342810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4a959-16c7-4075-bb45-8de0f8111f50 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:18.389819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76d752-8de2-419f-954d-4ac03602667d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:18.390469 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76d752-8de2-419f-954d-4ac03602667d None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:18.390754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76d752-8de2-419f-954d-4ac03602667d None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:18.391032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76d752-8de2-419f-954d-4ac03602667d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:18.391883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76d752-8de2-419f-954d-4ac03602667d None admin] RBAC: Adding query filter params (role.id=97c55e5f00404a29bb3f2df9566b6b0f, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=e3517b3b077a4a1fa76163657b46930c) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:18.392215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76d752-8de2-419f-954d-4ac03602667d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:18.393813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76d752-8de2-419f-954d-4ac03602667d None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:18.404796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 548/1110] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:18 2022] GET /identity/v3/role_assignments?user.id=e3517b3b077a4a1fa76163657b46930c&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=97c55e5f00404a29bb3f2df9566b6b0f => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:20.427254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be25125-1999-4309-ac0a-4c15de6ab878 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:20.427858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be25125-1999-4309-ac0a-4c15de6ab878 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:20.427858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be25125-1999-4309-ac0a-4c15de6ab878 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:20.429364 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 563/1111] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:20.436170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f750e-f6a1-4ca5-bb70-9c340352e0da None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:20.436820 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f750e-f6a1-4ca5-bb70-9c340352e0da None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:20.436820 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f750e-f6a1-4ca5-bb70-9c340352e0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:20.456541 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-a13f750e-f6a1-4ca5-bb70-9c340352e0da None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:20.457838 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a13f750e-f6a1-4ca5-bb70-9c340352e0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:20.498509 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:20.533525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 549/1112] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:20.811102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be25125-1999-4309-ac0a-4c15de6ab878 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:20.923188 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be25125-1999-4309-ac0a-4c15de6ab878 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:20.944313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:20.964954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6288b7-6486-48a5-bc0b-894f352413f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:20.966828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6288b7-6486-48a5-bc0b-894f352413f4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:20.967150 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6288b7-6486-48a5-bc0b-894f352413f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:20.967470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6288b7-6486-48a5-bc0b-894f352413f4 None admin] PATH_INFO: `/v3/roles/3c32f171f34345e4958d31e3940092d6` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:20.968966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6288b7-6486-48a5-bc0b-894f352413f4 None admin] RBAC: Authorizing `identity:get_role(role_id=3c32f171f34345e4958d31e3940092d6)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:20.970431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6288b7-6486-48a5-bc0b-894f352413f4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:20.971854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 564/1113] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:20 2022] GET /identity/v3/roles/3c32f171f34345e4958d31e3940092d6 => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:20.978119 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f750e-f6a1-4ca5-bb70-9c340352e0da None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:21.084599 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f750e-f6a1-4ca5-bb70-9c340352e0da None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:21.131083 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03397cbd-399a-4407-9d01-acd50b81c23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:21.132008 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03397cbd-399a-4407-9d01-acd50b81c23e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:21.132501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03397cbd-399a-4407-9d01-acd50b81c23e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:21.133003 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03397cbd-399a-4407-9d01-acd50b81c23e None admin] PATH_INFO: `/v3/users/22c615df24fa4b54a7f1818f2a52b0fc` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:21.150667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03397cbd-399a-4407-9d01-acd50b81c23e None admin] RBAC: Authorizing `identity:get_user(user_id=22c615df24fa4b54a7f1818f2a52b0fc)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:21.152237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03397cbd-399a-4407-9d01-acd50b81c23e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:21.154669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 550/1114] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:20 2022] GET /identity/v3/users/22c615df24fa4b54a7f1818f2a52b0fc => generated 286 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:21.160617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6288b7-6486-48a5-bc0b-894f352413f4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:21.267997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6288b7-6486-48a5-bc0b-894f352413f4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:21.305358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd902410-5a21-448f-a87e-d96a7091fe0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:21.306031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd902410-5a21-448f-a87e-d96a7091fe0a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:21.306352 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd902410-5a21-448f-a87e-d96a7091fe0a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:21.306666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd902410-5a21-448f-a87e-d96a7091fe0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:21.394917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:21.410877 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd902410-5a21-448f-a87e-d96a7091fe0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:21.412368 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd902410-5a21-448f-a87e-d96a7091fe0a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:21.515567 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 565/1115] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:21.521879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03397cbd-399a-4407-9d01-acd50b81c23e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:21.556303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:21.632885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03397cbd-399a-4407-9d01-acd50b81c23e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:21.669985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d869371-f35a-449e-acc8-5d84736a4b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:21.670695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d869371-f35a-449e-acc8-5d84736a4b9a None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:21.671020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d869371-f35a-449e-acc8-5d84736a4b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:21.671338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d869371-f35a-449e-acc8-5d84736a4b9a None admin] PATH_INFO: `/v3/projects/0b50f1329c4e4582ad7ed027748bd8e5` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:21.672772 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d869371-f35a-449e-acc8-5d84736a4b9a None admin] RBAC: Authorizing `identity:get_project(project_id=0b50f1329c4e4582ad7ed027748bd8e5)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:21.674412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d869371-f35a-449e-acc8-5d84736a4b9a None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:21.676295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 551/1116] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:11:21 2022] GET /identity/v3/projects/0b50f1329c4e4582ad7ed027748bd8e5 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:21.683031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd902410-5a21-448f-a87e-d96a7091fe0a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:21.787734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:21.806765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd902410-5a21-448f-a87e-d96a7091fe0a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:21.850624 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967d5ef-fc94-4ce0-a5e0-774c2a2ac969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:21.851265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967d5ef-fc94-4ce0-a5e0-774c2a2ac969 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:21.851552 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967d5ef-fc94-4ce0-a5e0-774c2a2ac969 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:21.851826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967d5ef-fc94-4ce0-a5e0-774c2a2ac969 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:21.852639 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967d5ef-fc94-4ce0-a5e0-774c2a2ac969 None admin] RBAC: Adding query filter params (role.id=3c32f171f34345e4958d31e3940092d6, scope.project.id=0b50f1329c4e4582ad7ed027748bd8e5, user.id=22c615df24fa4b54a7f1818f2a52b0fc) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:21.852956 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967d5ef-fc94-4ce0-a5e0-774c2a2ac969 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:21.854573 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967d5ef-fc94-4ce0-a5e0-774c2a2ac969 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:21.861895 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 566/1117] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:21 2022] GET /identity/v3/role_assignments?user.id=22c615df24fa4b54a7f1818f2a52b0fc&scope.project.id=0b50f1329c4e4582ad7ed027748bd8e5&role.id=3c32f171f34345e4958d31e3940092d6 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:24.197965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750db8e-e019-4651-bb1f-df8e0f84e01b None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:24.198870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750db8e-e019-4651-bb1f-df8e0f84e01b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:24.199109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750db8e-e019-4651-bb1f-df8e0f84e01b None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:24.201727 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 552/1118] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:24.207732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9790fdc-0210-4d97-b1bc-67196b100455 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:24.208309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9790fdc-0210-4d97-b1bc-67196b100455 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:24.208309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9790fdc-0210-4d97-b1bc-67196b100455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:24.232904 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c9790fdc-0210-4d97-b1bc-67196b100455 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:24.234216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c9790fdc-0210-4d97-b1bc-67196b100455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:24.317501 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 567/1119] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:24.611813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750db8e-e019-4651-bb1f-df8e0f84e01b None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:24.760652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:24.768053 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750db8e-e019-4651-bb1f-df8e0f84e01b None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:24.819243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177dbdc-1122-4d6e-aec8-da228192face None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:24.820121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177dbdc-1122-4d6e-aec8-da228192face None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:24.820415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177dbdc-1122-4d6e-aec8-da228192face None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:24.820718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177dbdc-1122-4d6e-aec8-da228192face None admin] PATH_INFO: `/v3/roles/3c32f171f34345e4958d31e3940092d6` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:24.823376 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177dbdc-1122-4d6e-aec8-da228192face None admin] RBAC: Authorizing `identity:get_role(role_id=3c32f171f34345e4958d31e3940092d6)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:24.826382 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177dbdc-1122-4d6e-aec8-da228192face None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:24.829104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 553/1120] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:24 2022] GET /identity/v3/roles/3c32f171f34345e4958d31e3940092d6 => generated 223 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:24.838891 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9790fdc-0210-4d97-b1bc-67196b100455 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:24.987099 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:25.024920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9790fdc-0210-4d97-b1bc-67196b100455 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:25.087075 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca88ecc-75bd-4306-b050-7c947dc9d606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:25.087812 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca88ecc-75bd-4306-b050-7c947dc9d606 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:25.088148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca88ecc-75bd-4306-b050-7c947dc9d606 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:25.088505 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca88ecc-75bd-4306-b050-7c947dc9d606 None admin] PATH_INFO: `/v3/users/22c615df24fa4b54a7f1818f2a52b0fc` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:25.090856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca88ecc-75bd-4306-b050-7c947dc9d606 None admin] RBAC: Authorizing `identity:get_user(user_id=22c615df24fa4b54a7f1818f2a52b0fc)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:25.093013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca88ecc-75bd-4306-b050-7c947dc9d606 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:25.096249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 568/1121] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:24 2022] GET /identity/v3/users/22c615df24fa4b54a7f1818f2a52b0fc => generated 286 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:25.102244 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177dbdc-1122-4d6e-aec8-da228192face None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:25.245031 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177dbdc-1122-4d6e-aec8-da228192face None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:25.285516 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e620c1f-efb9-489f-bd09-4df448be5e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:25.286151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e620c1f-efb9-489f-bd09-4df448be5e2c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:25.286431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e620c1f-efb9-489f-bd09-4df448be5e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:25.286706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e620c1f-efb9-489f-bd09-4df448be5e2c None admin] PATH_INFO: `/v3/projects/0b50f1329c4e4582ad7ed027748bd8e5` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:25.287955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e620c1f-efb9-489f-bd09-4df448be5e2c None admin] RBAC: Authorizing `identity:get_project(project_id=0b50f1329c4e4582ad7ed027748bd8e5)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:25.289422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e620c1f-efb9-489f-bd09-4df448be5e2c None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:25.291121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 554/1122] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:11:25 2022] GET /identity/v3/projects/0b50f1329c4e4582ad7ed027748bd8e5 => generated 311 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:25.298785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca88ecc-75bd-4306-b050-7c947dc9d606 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:25.424550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca88ecc-75bd-4306-b050-7c947dc9d606 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:25.466500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0df6e-fd82-4975-801a-adbd0d9ac4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:25.467175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0df6e-fd82-4975-801a-adbd0d9ac4be None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:25.467463 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0df6e-fd82-4975-801a-adbd0d9ac4be None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:25.467733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0df6e-fd82-4975-801a-adbd0d9ac4be None admin] PATH_INFO: `/v3/projects/0b50f1329c4e4582ad7ed027748bd8e5/users/22c615df24fa4b54a7f1818f2a52b0fc/roles/3c32f171f34345e4958d31e3940092d6` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:25.472384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0df6e-fd82-4975-801a-adbd0d9ac4be None admin] RBAC: Authorizing `identity:create_grant(project_id=0b50f1329c4e4582ad7ed027748bd8e5, user_id=22c615df24fa4b54a7f1818f2a52b0fc, role_id=3c32f171f34345e4958d31e3940092d6)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:25.474165 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0df6e-fd82-4975-801a-adbd0d9ac4be None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:25.490491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 569/1123] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:11:25 2022] PUT /identity/v3/projects/0b50f1329c4e4582ad7ed027748bd8e5/users/22c615df24fa4b54a7f1818f2a52b0fc/roles/3c32f171f34345e4958d31e3940092d6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:11:27.499873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653d79c-826a-4914-a11c-a2a3e3442a4e None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:27.500472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653d79c-826a-4914-a11c-a2a3e3442a4e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:27.500472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653d79c-826a-4914-a11c-a2a3e3442a4e None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:27.501946 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 555/1124] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:11:27.507667 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40ede7-cad6-415a-8aa3-7eb799e6c3e7 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:27.508202 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40ede7-cad6-415a-8aa3-7eb799e6c3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:27.508348 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40ede7-cad6-415a-8aa3-7eb799e6c3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:27.527207 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fe40ede7-cad6-415a-8aa3-7eb799e6c3e7 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:27.529022 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fe40ede7-cad6-415a-8aa3-7eb799e6c3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:27.531535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:27.651103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 570/1125] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:27.922064 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653d79c-826a-4914-a11c-a2a3e3442a4e None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:27.965713 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:28.042277 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653d79c-826a-4914-a11c-a2a3e3442a4e None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:28.078378 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8eb3d-4717-4693-bc6d-ed03a494e334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:28.078923 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8eb3d-4717-4693-bc6d-ed03a494e334 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:28.079201 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8eb3d-4717-4693-bc6d-ed03a494e334 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:28.079472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8eb3d-4717-4693-bc6d-ed03a494e334 None admin] PATH_INFO: `/v3/roles/3c32f171f34345e4958d31e3940092d6` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:28.080876 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8eb3d-4717-4693-bc6d-ed03a494e334 None admin] RBAC: Authorizing `identity:get_role(role_id=3c32f171f34345e4958d31e3940092d6)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:28.082300 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8eb3d-4717-4693-bc6d-ed03a494e334 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:28.083696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 556/1126] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:27 2022] GET /identity/v3/roles/3c32f171f34345e4958d31e3940092d6 => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:28.089803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40ede7-cad6-415a-8aa3-7eb799e6c3e7 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:28.198982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40ede7-cad6-415a-8aa3-7eb799e6c3e7 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:28.237923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324b393-8373-42a0-a190-b77a41323dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:28.238649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324b393-8373-42a0-a190-b77a41323dba None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:28.238973 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324b393-8373-42a0-a190-b77a41323dba None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:28.239289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324b393-8373-42a0-a190-b77a41323dba None admin] PATH_INFO: `/v3/users/22c615df24fa4b54a7f1818f2a52b0fc` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:28.240538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324b393-8373-42a0-a190-b77a41323dba None admin] RBAC: Authorizing `identity:get_user(user_id=22c615df24fa4b54a7f1818f2a52b0fc)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:28.242089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324b393-8373-42a0-a190-b77a41323dba None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:28.243752 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 571/1127] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:28 2022] GET /identity/v3/users/22c615df24fa4b54a7f1818f2a52b0fc => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:28.251233 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8eb3d-4717-4693-bc6d-ed03a494e334 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:28.379532 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8eb3d-4717-4693-bc6d-ed03a494e334 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:28.381385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:28.427596 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbb900-6e86-4a73-b7dc-d5cd69c1eaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:28.428352 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbb900-6e86-4a73-b7dc-d5cd69c1eaf8 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:28.428731 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbb900-6e86-4a73-b7dc-d5cd69c1eaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:28.429090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbb900-6e86-4a73-b7dc-d5cd69c1eaf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:28.513774 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbb900-6e86-4a73-b7dc-d5cd69c1eaf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:28.515308 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbb900-6e86-4a73-b7dc-d5cd69c1eaf8 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:28.667325 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 557/1128] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:28.674955 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324b393-8373-42a0-a190-b77a41323dba None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:28.691900 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:28.866709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324b393-8373-42a0-a190-b77a41323dba None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:28.931279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3ada2-8849-4e12-b6cc-3ebd1ee25157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:28.932286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3ada2-8849-4e12-b6cc-3ebd1ee25157 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:28.932782 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3ada2-8849-4e12-b6cc-3ebd1ee25157 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:28.933326 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3ada2-8849-4e12-b6cc-3ebd1ee25157 None admin] PATH_INFO: `/v3/projects/0b50f1329c4e4582ad7ed027748bd8e5` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:28.936148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3ada2-8849-4e12-b6cc-3ebd1ee25157 None admin] RBAC: Authorizing `identity:get_project(project_id=0b50f1329c4e4582ad7ed027748bd8e5)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:28.938984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3ada2-8849-4e12-b6cc-3ebd1ee25157 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:28.942270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 572/1129] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:11:28 2022] GET /identity/v3/projects/0b50f1329c4e4582ad7ed027748bd8e5 => generated 311 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:28.948811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbb900-6e86-4a73-b7dc-d5cd69c1eaf8 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:28.952148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:29.090688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbb900-6e86-4a73-b7dc-d5cd69c1eaf8 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:29.138972 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19a316-541b-4f5e-9cea-419839a6f002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:29.139892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19a316-541b-4f5e-9cea-419839a6f002 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:29.140216 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19a316-541b-4f5e-9cea-419839a6f002 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:29.140629 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19a316-541b-4f5e-9cea-419839a6f002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:29.141735 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19a316-541b-4f5e-9cea-419839a6f002 None admin] RBAC: Adding query filter params (role.id=3c32f171f34345e4958d31e3940092d6, scope.project.id=0b50f1329c4e4582ad7ed027748bd8e5, user.id=22c615df24fa4b54a7f1818f2a52b0fc) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:29.142817 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19a316-541b-4f5e-9cea-419839a6f002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:29.145048 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19a316-541b-4f5e-9cea-419839a6f002 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:29.157839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 558/1130] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:28 2022] GET /identity/v3/role_assignments?user.id=22c615df24fa4b54a7f1818f2a52b0fc&scope.project.id=0b50f1329c4e4582ad7ed027748bd8e5&role.id=3c32f171f34345e4958d31e3940092d6 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:31.294255 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b6034-67cb-4b35-b99e-54761f28ccdb None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:31.294861 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b6034-67cb-4b35-b99e-54761f28ccdb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:31.294861 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b6034-67cb-4b35-b99e-54761f28ccdb None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:31.296151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 573/1131] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:31.301715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3210c-cf51-4bcd-81fd-a954ffba20ee None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:31.302218 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3210c-cf51-4bcd-81fd-a954ffba20ee None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:31.302323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3210c-cf51-4bcd-81fd-a954ffba20ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:31.320518 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-77c3210c-cf51-4bcd-81fd-a954ffba20ee None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:31.321752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-77c3210c-cf51-4bcd-81fd-a954ffba20ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:31.382718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:31.391333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 559/1132] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:31.662830 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b6034-67cb-4b35-b99e-54761f28ccdb None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:31.739955 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:31.767381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b6034-67cb-4b35-b99e-54761f28ccdb None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:31.803429 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526bfdb-65b8-4246-8b2e-baf3f09b0458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:31.804062 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526bfdb-65b8-4246-8b2e-baf3f09b0458 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:31.804342 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526bfdb-65b8-4246-8b2e-baf3f09b0458 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:31.804614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526bfdb-65b8-4246-8b2e-baf3f09b0458 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:31.805735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526bfdb-65b8-4246-8b2e-baf3f09b0458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:31.807160 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526bfdb-65b8-4246-8b2e-baf3f09b0458 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:31.809545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 574/1133] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:31.815611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3210c-cf51-4bcd-81fd-a954ffba20ee None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:31.916109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3210c-cf51-4bcd-81fd-a954ffba20ee None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:31.951635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f5bdd-eef9-4280-9e19-9a7900f0a6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:31.952243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f5bdd-eef9-4280-9e19-9a7900f0a6f2 None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:31.952523 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f5bdd-eef9-4280-9e19-9a7900f0a6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:31.952792 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f5bdd-eef9-4280-9e19-9a7900f0a6f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:31.953465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f5bdd-eef9-4280-9e19-9a7900f0a6f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:31.954898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f5bdd-eef9-4280-9e19-9a7900f0a6f2 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:31.981282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 560/1134] 10.222.0.88 () {60 vars in 1141 bytes} [Tue Aug 23 22:11:31 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:33.958168 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684638b5-439e-4772-b2ff-ba8f148834e7 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:33.958732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684638b5-439e-4772-b2ff-ba8f148834e7 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:33.958732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684638b5-439e-4772-b2ff-ba8f148834e7 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:33.960117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 575/1135] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:33.967358 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d215575-a13d-4409-b50b-4636e365fc6d None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:33.967927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d215575-a13d-4409-b50b-4636e365fc6d None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:33.967927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d215575-a13d-4409-b50b-4636e365fc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:33.987862 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-1d215575-a13d-4409-b50b-4636e365fc6d None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:33.989342 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-1d215575-a13d-4409-b50b-4636e365fc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:34.053868 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:34.081803 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 561/1136] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:34.360361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684638b5-439e-4772-b2ff-ba8f148834e7 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:34.483778 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684638b5-439e-4772-b2ff-ba8f148834e7 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:34.521729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c899a-5fa1-480a-ad6d-39df3da2d7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:34.522376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c899a-5fa1-480a-ad6d-39df3da2d7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:34.522654 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c899a-5fa1-480a-ad6d-39df3da2d7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:34.522926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c899a-5fa1-480a-ad6d-39df3da2d7b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:34.524095 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c899a-5fa1-480a-ad6d-39df3da2d7b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:34.525593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c899a-5fa1-480a-ad6d-39df3da2d7b8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:34.527862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 576/1137] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:34.534208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d215575-a13d-4409-b50b-4636e365fc6d None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:34.648591 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d215575-a13d-4409-b50b-4636e365fc6d None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:34.683834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04fd67f-209c-4d7b-a793-36197f845e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:34.684438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fd67f-209c-4d7b-a793-36197f845e50 None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:34.684714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fd67f-209c-4d7b-a793-36197f845e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:34.684980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fd67f-209c-4d7b-a793-36197f845e50 None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:34.685915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fd67f-209c-4d7b-a793-36197f845e50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:34.687350 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fd67f-209c-4d7b-a793-36197f845e50 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:34.709363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 562/1138] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:11:34 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:11:36.695431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212dc57-2794-42bb-b683-5e10036f99fd None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:36.696012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212dc57-2794-42bb-b683-5e10036f99fd None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:36.696012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212dc57-2794-42bb-b683-5e10036f99fd None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:36.697347 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 577/1139] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:36.703464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f580b3-a870-4bb0-a3d1-fb2efebba7c9 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:36.703911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f580b3-a870-4bb0-a3d1-fb2efebba7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:36.703999 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f580b3-a870-4bb0-a3d1-fb2efebba7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:36.722638 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-c2f580b3-a870-4bb0-a3d1-fb2efebba7c9 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:36.723832 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c2f580b3-a870-4bb0-a3d1-fb2efebba7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:36.752172 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:36.795702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 563/1140] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:37.077813 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212dc57-2794-42bb-b683-5e10036f99fd None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:37.091520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:37.188976 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212dc57-2794-42bb-b683-5e10036f99fd None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:37.225259 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9147233-1836-4bea-b079-b5614487204a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:37.225918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9147233-1836-4bea-b079-b5614487204a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:37.226203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9147233-1836-4bea-b079-b5614487204a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:37.226477 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9147233-1836-4bea-b079-b5614487204a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:37.238128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9147233-1836-4bea-b079-b5614487204a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:37.242995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-c9147233-1836-4bea-b079-b5614487204a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:37.242995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:37.242995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:37.242995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:37.242995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:37.242995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:37.242995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:37.242995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:37.242995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:37.242995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:37.242995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:37.242995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:37.242995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:37.242995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:37.242995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:37.242995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:37.244203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:37.244203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:37.244203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:37.244203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:37.244203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:37.244203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:37.244203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:37.244203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:37.244203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:37.244203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:37.244203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:37.244203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:37.244203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:37.244203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:37.244203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:37.244203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:37.245183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:37.245183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:37.245183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:37.245183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:37.245183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:37.245183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:37.245183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:37.245183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:37.245183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:37.245183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:37.245183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:37.245183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:37.245183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 578/1141] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:11:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:37.250744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f580b3-a870-4bb0-a3d1-fb2efebba7c9 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:37.358758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f580b3-a870-4bb0-a3d1-fb2efebba7c9 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:37.393786 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b20ef-cc7a-4a80-b3cb-c3e5534c5dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:37.394401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b20ef-cc7a-4a80-b3cb-c3e5534c5dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:37.394679 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b20ef-cc7a-4a80-b3cb-c3e5534c5dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:37.394954 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b20ef-cc7a-4a80-b3cb-c3e5534c5dc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:37.395625 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b20ef-cc7a-4a80-b3cb-c3e5534c5dc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:37.395931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b20ef-cc7a-4a80-b3cb-c3e5534c5dc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:37.397343 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b20ef-cc7a-4a80-b3cb-c3e5534c5dc9 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:37.407297 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 564/1142] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:11:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:37.413302 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9147233-1836-4bea-b079-b5614487204a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:37.483129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:37.515235 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9147233-1836-4bea-b079-b5614487204a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:37.549115 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6380c4-3d35-446c-aff7-96934c189e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:37.549743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6380c4-3d35-446c-aff7-96934c189e06 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:37.550020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6380c4-3d35-446c-aff7-96934c189e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:37.550287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6380c4-3d35-446c-aff7-96934c189e06 None admin] PATH_INFO: `/v3/users/3948d3c4b21f499396effc81ffa73e5f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:37.566600 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6380c4-3d35-446c-aff7-96934c189e06 None admin] RBAC: Authorizing `identity:get_user(user_id=3948d3c4b21f499396effc81ffa73e5f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:37.569135 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6380c4-3d35-446c-aff7-96934c189e06 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:37.570875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 579/1143] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:37 2022] GET /identity/v3/users/3948d3c4b21f499396effc81ffa73e5f => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:37.577651 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b20ef-cc7a-4a80-b3cb-c3e5534c5dc9 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:37.605481 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:37.680337 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b20ef-cc7a-4a80-b3cb-c3e5534c5dc9 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:37.735931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28b947-1f63-45dc-af02-a945aecc7988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:37.736592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28b947-1f63-45dc-af02-a945aecc7988 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:37.736811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28b947-1f63-45dc-af02-a945aecc7988 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:37.737108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28b947-1f63-45dc-af02-a945aecc7988 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:37.808887 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28b947-1f63-45dc-af02-a945aecc7988 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:37.810836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28b947-1f63-45dc-af02-a945aecc7988 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:37.837192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:37.915517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 565/1144] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:37.922336 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6380c4-3d35-446c-aff7-96934c189e06 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:38.033470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6380c4-3d35-446c-aff7-96934c189e06 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:38.060906 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:38.069471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dd54e-2fc9-4eeb-9761-c53caf4a67b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:38.070106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dd54e-2fc9-4eeb-9761-c53caf4a67b2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:38.070390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dd54e-2fc9-4eeb-9761-c53caf4a67b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:38.070662 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dd54e-2fc9-4eeb-9761-c53caf4a67b2 None admin] PATH_INFO: `/v3/projects/9636d23fac3d473695dba1e578a41b51` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:38.071805 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0dd54e-2fc9-4eeb-9761-c53caf4a67b2 None admin] RBAC: Authorizing `identity:get_project(project_id=9636d23fac3d473695dba1e578a41b51)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:38.073287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0dd54e-2fc9-4eeb-9761-c53caf4a67b2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:38.075681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 580/1145] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:11:37 2022] GET /identity/v3/projects/9636d23fac3d473695dba1e578a41b51 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:38.082002 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28b947-1f63-45dc-af02-a945aecc7988 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:38.182938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28b947-1f63-45dc-af02-a945aecc7988 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:38.210462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:38.220380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bc9e4-c624-4d60-9ddd-88fe19ae3cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:38.220989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bc9e4-c624-4d60-9ddd-88fe19ae3cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:38.221308 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bc9e4-c624-4d60-9ddd-88fe19ae3cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:38.221580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bc9e4-c624-4d60-9ddd-88fe19ae3cc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:38.222319 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bc9e4-c624-4d60-9ddd-88fe19ae3cc3 None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=9636d23fac3d473695dba1e578a41b51, user.id=3948d3c4b21f499396effc81ffa73e5f) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:38.222625 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bc9e4-c624-4d60-9ddd-88fe19ae3cc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:38.224069 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bc9e4-c624-4d60-9ddd-88fe19ae3cc3 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:38.229915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 566/1146] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:38 2022] GET /identity/v3/role_assignments?user.id=3948d3c4b21f499396effc81ffa73e5f&scope.project.id=9636d23fac3d473695dba1e578a41b51&role.id=dc5d71ab685e420697f520aa3a368177 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:40.275160 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242aa6-3930-4cdd-a1c5-3794a015ddbf None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:40.275739 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242aa6-3930-4cdd-a1c5-3794a015ddbf None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:40.275739 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242aa6-3930-4cdd-a1c5-3794a015ddbf None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:40.278095 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 581/1147] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:40.284137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff86da3-7030-47f2-903f-bb0359ec62c5 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:40.284695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff86da3-7030-47f2-903f-bb0359ec62c5 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:40.284695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff86da3-7030-47f2-903f-bb0359ec62c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:40.304073 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5ff86da3-7030-47f2-903f-bb0359ec62c5 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:40.305358 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5ff86da3-7030-47f2-903f-bb0359ec62c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:40.388086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 567/1148] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:40.670648 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea242aa6-3930-4cdd-a1c5-3794a015ddbf None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:40.780786 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea242aa6-3930-4cdd-a1c5-3794a015ddbf None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:40.818212 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc93851-afde-4176-9f05-1a85355ba4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:40.818847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc93851-afde-4176-9f05-1a85355ba4e8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:40.819129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc93851-afde-4176-9f05-1a85355ba4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:40.819405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc93851-afde-4176-9f05-1a85355ba4e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:40.830767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc93851-afde-4176-9f05-1a85355ba4e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:40.835545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-0dc93851-afde-4176-9f05-1a85355ba4e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:40.835545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:40.835545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:40.835545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:40.835545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:40.835545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:40.835545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:40.835545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:40.835545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:40.835545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:40.835545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:40.835545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:40.835545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:40.835545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:40.835545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:40.835545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:40.836647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:40.836647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:40.836647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:40.836647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:40.836647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:40.836647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:40.836647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:40.836647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:40.836647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:40.836647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:40.836647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:40.836647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:40.836647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:40.836647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:40.836647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:40.836647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:40.837533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:40.837533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:40.837533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:40.837533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:40.837533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:40.837533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:40.837533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:40.837533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:40.837533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:40.837533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:40.837533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:40.837533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:40.837533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 582/1149] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:11:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:40.843729 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff86da3-7030-47f2-903f-bb0359ec62c5 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:40.945702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff86da3-7030-47f2-903f-bb0359ec62c5 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:40.967788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:40.983194 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52271ad-c5a5-4c52-bc64-3992bd99bb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:40.983808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52271ad-c5a5-4c52-bc64-3992bd99bb0f None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:40.984086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52271ad-c5a5-4c52-bc64-3992bd99bb0f None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:40.984357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52271ad-c5a5-4c52-bc64-3992bd99bb0f None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:40.985024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52271ad-c5a5-4c52-bc64-3992bd99bb0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:40.985358 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52271ad-c5a5-4c52-bc64-3992bd99bb0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:40.986730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52271ad-c5a5-4c52-bc64-3992bd99bb0f None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:40.995205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 568/1150] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:11:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:41.001877 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc93851-afde-4176-9f05-1a85355ba4e8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:41.066271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:41.114478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc93851-afde-4176-9f05-1a85355ba4e8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:41.150062 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a63cdd-8b6b-412c-9504-c53e0588307e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:41.150690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a63cdd-8b6b-412c-9504-c53e0588307e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:41.150974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a63cdd-8b6b-412c-9504-c53e0588307e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:41.151247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a63cdd-8b6b-412c-9504-c53e0588307e None admin] PATH_INFO: `/v3/users/3948d3c4b21f499396effc81ffa73e5f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:41.153161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a63cdd-8b6b-412c-9504-c53e0588307e None admin] RBAC: Authorizing `identity:get_user(user_id=3948d3c4b21f499396effc81ffa73e5f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:41.154650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a63cdd-8b6b-412c-9504-c53e0588307e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:41.156430 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 583/1151] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:41 2022] GET /identity/v3/users/3948d3c4b21f499396effc81ffa73e5f => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:41.162709 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52271ad-c5a5-4c52-bc64-3992bd99bb0f None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:41.265725 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52271ad-c5a5-4c52-bc64-3992bd99bb0f None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:41.301571 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fa52c-92b9-4763-806a-645d760c5630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:41.302283 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fa52c-92b9-4763-806a-645d760c5630 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:41.302473 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fa52c-92b9-4763-806a-645d760c5630 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:41.302748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fa52c-92b9-4763-806a-645d760c5630 None admin] PATH_INFO: `/v3/projects/9636d23fac3d473695dba1e578a41b51` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:41.303904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fa52c-92b9-4763-806a-645d760c5630 None admin] RBAC: Authorizing `identity:get_project(project_id=9636d23fac3d473695dba1e578a41b51)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:41.305376 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fa52c-92b9-4763-806a-645d760c5630 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:41.307070 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 569/1152] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:11:41 2022] GET /identity/v3/projects/9636d23fac3d473695dba1e578a41b51 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:41.313532 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a63cdd-8b6b-412c-9504-c53e0588307e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:41.421431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a63cdd-8b6b-412c-9504-c53e0588307e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:41.446214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:41.458163 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c246ebd-5a15-4eff-8dcb-a441c278960c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:41.458818 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c246ebd-5a15-4eff-8dcb-a441c278960c None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:41.459094 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c246ebd-5a15-4eff-8dcb-a441c278960c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:41.459365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c246ebd-5a15-4eff-8dcb-a441c278960c None admin] PATH_INFO: `/v3/projects/9636d23fac3d473695dba1e578a41b51/users/3948d3c4b21f499396effc81ffa73e5f/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:41.461397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c246ebd-5a15-4eff-8dcb-a441c278960c None admin] RBAC: Authorizing `identity:create_grant(project_id=9636d23fac3d473695dba1e578a41b51, user_id=3948d3c4b21f499396effc81ffa73e5f, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:41.462835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c246ebd-5a15-4eff-8dcb-a441c278960c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:41.477542 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 584/1153] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:11:41 2022] PUT /identity/v3/projects/9636d23fac3d473695dba1e578a41b51/users/3948d3c4b21f499396effc81ffa73e5f/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:11:43.463608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9b88f-035c-446f-8301-ca1530c3b855 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:43.464181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9b88f-035c-446f-8301-ca1530c3b855 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:43.464181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9b88f-035c-446f-8301-ca1530c3b855 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:43.466666 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 570/1154] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:43.473888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d998f4f-8475-46c4-be04-214752b34438 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:43.474445 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d998f4f-8475-46c4-be04-214752b34438 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:43.474837 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d998f4f-8475-46c4-be04-214752b34438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:43.496739 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7d998f4f-8475-46c4-be04-214752b34438 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:43.498032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7d998f4f-8475-46c4-be04-214752b34438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:43.578193 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 585/1155] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:43.864285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9b88f-035c-446f-8301-ca1530c3b855 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:43.925298 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:43.991240 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9b88f-035c-446f-8301-ca1530c3b855 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:44.034974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c88693-fa21-43b5-aedd-69428378ab0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:44.035767 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c88693-fa21-43b5-aedd-69428378ab0c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:44.036137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c88693-fa21-43b5-aedd-69428378ab0c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:44.036514 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c88693-fa21-43b5-aedd-69428378ab0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:44.051895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c88693-fa21-43b5-aedd-69428378ab0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:44.057269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-e9c88693-fa21-43b5-aedd-69428378ab0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:44.057269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:44.057269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:44.057269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:44.057269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:44.057269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:44.057269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:44.057269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:44.057269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:44.057269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:44.057269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:44.057269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:44.057269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:44.057269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:44.057269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:44.057269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:44.059785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:44.059785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:44.059785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:44.059785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:44.059785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:44.059785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:44.059785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:44.059785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:44.059785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:44.059785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:44.059785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:44.059785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:44.059785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:44.059785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:44.059785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:44.059785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:44.061986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:44.061986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:44.061986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:44.061986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:44.061986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:44.061986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:44.061986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:44.061986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:44.061986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:44.061986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:44.061986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:44.061986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:11:44.061986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 571/1156] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:11:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:44.068052 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d998f4f-8475-46c4-be04-214752b34438 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:44.255400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d998f4f-8475-46c4-be04-214752b34438 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:44.284304 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:44.321697 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f724581-2454-41f9-a3b7-f235e232b872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:44.322641 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f724581-2454-41f9-a3b7-f235e232b872 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:44.323137 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f724581-2454-41f9-a3b7-f235e232b872 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:44.323658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f724581-2454-41f9-a3b7-f235e232b872 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:44.325113 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f724581-2454-41f9-a3b7-f235e232b872 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:44.325669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f724581-2454-41f9-a3b7-f235e232b872 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:44.328272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f724581-2454-41f9-a3b7-f235e232b872 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:44.344599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 586/1157] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:11:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:44.350993 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c88693-fa21-43b5-aedd-69428378ab0c None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:44.470895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c88693-fa21-43b5-aedd-69428378ab0c None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:44.483831 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:44.512512 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5483ef-b712-4874-babf-ff2f034ce23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:44.513319 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5483ef-b712-4874-babf-ff2f034ce23c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:44.513602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5483ef-b712-4874-babf-ff2f034ce23c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:44.513873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5483ef-b712-4874-babf-ff2f034ce23c None admin] PATH_INFO: `/v3/users/3948d3c4b21f499396effc81ffa73e5f` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:44.515109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5483ef-b712-4874-babf-ff2f034ce23c None admin] RBAC: Authorizing `identity:get_user(user_id=3948d3c4b21f499396effc81ffa73e5f)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:44.516553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5483ef-b712-4874-babf-ff2f034ce23c None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:44.518215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 572/1158] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:44 2022] GET /identity/v3/users/3948d3c4b21f499396effc81ffa73e5f => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:44.525335 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f724581-2454-41f9-a3b7-f235e232b872 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:44.642718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f724581-2454-41f9-a3b7-f235e232b872 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:44.681338 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dad40-0353-4ac9-8ba7-85c7a92c6ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:44.682082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dad40-0353-4ac9-8ba7-85c7a92c6ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:44.682504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dad40-0353-4ac9-8ba7-85c7a92c6ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:44.682894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dad40-0353-4ac9-8ba7-85c7a92c6ea8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:44.742731 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:44.780044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6dad40-0353-4ac9-8ba7-85c7a92c6ea8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:44.782753 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6dad40-0353-4ac9-8ba7-85c7a92c6ea8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:44.939555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 587/1159] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:44.945867 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5483ef-b712-4874-babf-ff2f034ce23c None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:45.109776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5483ef-b712-4874-babf-ff2f034ce23c None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:45.155750 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa894d45-460e-4ee3-a2ab-4cbf1daa0515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:45.156431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa894d45-460e-4ee3-a2ab-4cbf1daa0515 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:45.156762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa894d45-460e-4ee3-a2ab-4cbf1daa0515 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:45.157121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa894d45-460e-4ee3-a2ab-4cbf1daa0515 None admin] PATH_INFO: `/v3/projects/9636d23fac3d473695dba1e578a41b51` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:45.158366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa894d45-460e-4ee3-a2ab-4cbf1daa0515 None admin] RBAC: Authorizing `identity:get_project(project_id=9636d23fac3d473695dba1e578a41b51)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:45.159875 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa894d45-460e-4ee3-a2ab-4cbf1daa0515 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:45.161682 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 573/1160] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:11:44 2022] GET /identity/v3/projects/9636d23fac3d473695dba1e578a41b51 => generated 311 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:45.172864 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dad40-0353-4ac9-8ba7-85c7a92c6ea8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:45.277758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:45.306461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dad40-0353-4ac9-8ba7-85c7a92c6ea8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:45.347318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9e2ee-6c49-4fd3-af0b-02a7ed39dc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:45.348056 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9e2ee-6c49-4fd3-af0b-02a7ed39dc15 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:45.348437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9e2ee-6c49-4fd3-af0b-02a7ed39dc15 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:45.348802 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9e2ee-6c49-4fd3-af0b-02a7ed39dc15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:45.349692 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9e2ee-6c49-4fd3-af0b-02a7ed39dc15 None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=9636d23fac3d473695dba1e578a41b51, user.id=3948d3c4b21f499396effc81ffa73e5f) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:45.350105 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9e2ee-6c49-4fd3-af0b-02a7ed39dc15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:45.351676 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9e2ee-6c49-4fd3-af0b-02a7ed39dc15 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:45.359697 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 588/1161] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:45 2022] GET /identity/v3/role_assignments?user.id=3948d3c4b21f499396effc81ffa73e5f&scope.project.id=9636d23fac3d473695dba1e578a41b51&role.id=dc5d71ab685e420697f520aa3a368177 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:47.484219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe5dc4-489b-42d0-afdd-fc25e577667a None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:47.485145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe5dc4-489b-42d0-afdd-fc25e577667a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:47.485145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe5dc4-489b-42d0-afdd-fc25e577667a None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:47.486435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 574/1162] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:11:47.492774 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e761e-5266-4cf0-85a6-01f215fe20bc None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:47.493685 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e761e-5266-4cf0-85a6-01f215fe20bc None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:47.493685 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e761e-5266-4cf0-85a6-01f215fe20bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:47.515290 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-3c8e761e-5266-4cf0-85a6-01f215fe20bc None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:47.516600 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3c8e761e-5266-4cf0-85a6-01f215fe20bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:47.601354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 589/1163] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:47.880858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe5dc4-489b-42d0-afdd-fc25e577667a None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:47.973152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:48.049704 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe5dc4-489b-42d0-afdd-fc25e577667a None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:48.100585 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa49ea-4013-459d-a05d-4b29b998c2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:48.101260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa49ea-4013-459d-a05d-4b29b998c2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:48.101546 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa49ea-4013-459d-a05d-4b29b998c2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:48.101822 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa49ea-4013-459d-a05d-4b29b998c2f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:48.192405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa49ea-4013-459d-a05d-4b29b998c2f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:48.194126 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa49ea-4013-459d-a05d-4b29b998c2f8 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:48.263232 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:48.334212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 575/1164] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:48.341334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e761e-5266-4cf0-85a6-01f215fe20bc None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:48.430851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:48.469275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e761e-5266-4cf0-85a6-01f215fe20bc None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:48.512869 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f705ee-c3b5-426b-93da-b8c6d74d2847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:48.513741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f705ee-c3b5-426b-93da-b8c6d74d2847 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:48.513883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f705ee-c3b5-426b-93da-b8c6d74d2847 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:48.514105 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f705ee-c3b5-426b-93da-b8c6d74d2847 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:48.522182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f705ee-c3b5-426b-93da-b8c6d74d2847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:48.523730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f705ee-c3b5-426b-93da-b8c6d74d2847 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-45f705ee-c3b5-426b-93da-b8c6d74d2847 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:48.537220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:48.539241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:48.539241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:48.539241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:48.539241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:48.539241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:48.539241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:48.539241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:48.539241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:48.539241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:48.539241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:48.539241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:48.539241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:48.539241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:48.539241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:48.539241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:48.539241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:48.541155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:48.541155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:48.541155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:48.541155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:48.541155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:48.541155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:48.541155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:48.541155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:48.541155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:48.541155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:48.541155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 23 22:11:48.541155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:48.541155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 23 22:11:48.541155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:11:48.541155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 590/1165] 10.222.0.88 () {56 vars in 1117 bytes} [Tue Aug 23 22:11:48 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:48.547595 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa49ea-4013-459d-a05d-4b29b998c2f8 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:48.680963 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa49ea-4013-459d-a05d-4b29b998c2f8 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:48.731452 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9010370-3fbd-4036-b707-f916f3331cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:48.732077 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9010370-3fbd-4036-b707-f916f3331cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:48.732355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9010370-3fbd-4036-b707-f916f3331cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:48.732629 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9010370-3fbd-4036-b707-f916f3331cc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:48.733359 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9010370-3fbd-4036-b707-f916f3331cc0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:48.733662 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9010370-3fbd-4036-b707-f916f3331cc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:48.735099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9010370-3fbd-4036-b707-f916f3331cc0 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:48.743874 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 576/1166] 10.222.0.88 () {56 vars in 1115 bytes} [Tue Aug 23 22:11:48 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:50.798385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17038c92-77e2-4676-a4b7-30c54166dae2 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:50.799031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17038c92-77e2-4676-a4b7-30c54166dae2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:50.799031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17038c92-77e2-4676-a4b7-30c54166dae2 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:50.800270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 591/1167] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:50.806217 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37aa026-a38f-42ce-a14b-e2a25439990d None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:50.806751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37aa026-a38f-42ce-a14b-e2a25439990d None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:50.806751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37aa026-a38f-42ce-a14b-e2a25439990d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:50.828303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-f37aa026-a38f-42ce-a14b-e2a25439990d None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:50.829590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f37aa026-a38f-42ce-a14b-e2a25439990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:50.832981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:50.909197 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 577/1168] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:51.183183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17038c92-77e2-4676-a4b7-30c54166dae2 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:51.293828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17038c92-77e2-4676-a4b7-30c54166dae2 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:51.324059 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:51.330910 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47459c-4dfd-466a-b1cd-6d8c42803216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:51.331519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47459c-4dfd-466a-b1cd-6d8c42803216 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:51.331800 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47459c-4dfd-466a-b1cd-6d8c42803216 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:51.332076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47459c-4dfd-466a-b1cd-6d8c42803216 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:51.332755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47459c-4dfd-466a-b1cd-6d8c42803216 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:51.334170 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47459c-4dfd-466a-b1cd-6d8c42803216 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:51.352389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 592/1169] 10.222.0.88 () {60 vars in 1138 bytes} [Tue Aug 23 22:11:51 2022] POST /identity/v3/domains => generated 267 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:53.473860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46166e9a-96e3-4227-89b8-ce0e937bb80a None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:53.474823 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46166e9a-96e3-4227-89b8-ce0e937bb80a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:53.474823 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46166e9a-96e3-4227-89b8-ce0e937bb80a None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:53.476279 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 578/1170] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:53.482499 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeb0d3-bb8a-435a-bad1-223c13c75fae None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:53.482997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeb0d3-bb8a-435a-bad1-223c13c75fae None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:53.483107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeb0d3-bb8a-435a-bad1-223c13c75fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:53.503664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-3eeeb0d3-bb8a-435a-bad1-223c13c75fae None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:53.504915 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3eeeb0d3-bb8a-435a-bad1-223c13c75fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:53.583616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 593/1171] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:53.860420 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46166e9a-96e3-4227-89b8-ce0e937bb80a None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:53.969446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46166e9a-96e3-4227-89b8-ce0e937bb80a None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:54.005979 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c78b9-ab5c-4c45-861d-ebd401febaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:54.006615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c78b9-ab5c-4c45-861d-ebd401febaa2 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:54.006892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c78b9-ab5c-4c45-861d-ebd401febaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:54.007166 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c78b9-ab5c-4c45-861d-ebd401febaa2 None admin] PATH_INFO: `/v3/domains/f67184c979d84a80aab0131c0e8417b7` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:54.016298 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:54.019202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c78b9-ab5c-4c45-861d-ebd401febaa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f67184c979d84a80aab0131c0e8417b7)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:54.020857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c78b9-ab5c-4c45-861d-ebd401febaa2 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:54.022672 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 579/1172] 10.222.0.88 () {56 vars in 1183 bytes} [Tue Aug 23 22:11:53 2022] GET /identity/v3/domains/f67184c979d84a80aab0131c0e8417b7 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:54.028428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeb0d3-bb8a-435a-bad1-223c13c75fae None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:54.131069 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeb0d3-bb8a-435a-bad1-223c13c75fae None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:54.151821 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:54.166077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a347334-e04a-429c-b2a9-65b810dbab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:54.166683 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a347334-e04a-429c-b2a9-65b810dbab49 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:54.167052 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a347334-e04a-429c-b2a9-65b810dbab49 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:54.167825 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a347334-e04a-429c-b2a9-65b810dbab49 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:54.168463 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a347334-e04a-429c-b2a9-65b810dbab49 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:54.169925 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a347334-e04a-429c-b2a9-65b810dbab49 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:54.196818 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 594/1173] 10.222.0.88 () {60 vars in 1141 bytes} [Tue Aug 23 22:11:54 2022] POST /identity/v3/projects => generated 361 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:56.186642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46344f31-936a-4c39-84cd-05b2908a0ec2 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:56.187495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46344f31-936a-4c39-84cd-05b2908a0ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:56.187495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46344f31-936a-4c39-84cd-05b2908a0ec2 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:56.188529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 580/1174] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:11:56.195336 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1158f77-5ec9-4bfc-93f6-76de8e0e208c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:56.195936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1158f77-5ec9-4bfc-93f6-76de8e0e208c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:56.196324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1158f77-5ec9-4bfc-93f6-76de8e0e208c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:56.218456 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a1158f77-5ec9-4bfc-93f6-76de8e0e208c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:56.219755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a1158f77-5ec9-4bfc-93f6-76de8e0e208c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:56.307380 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 595/1175] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:56.586575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46344f31-936a-4c39-84cd-05b2908a0ec2 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:56.708302 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46344f31-936a-4c39-84cd-05b2908a0ec2 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:56.773286 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b42366-e365-4415-b576-e59d8a4405d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:56.774261 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b42366-e365-4415-b576-e59d8a4405d3 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:56.774684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b42366-e365-4415-b576-e59d8a4405d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:56.775095 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b42366-e365-4415-b576-e59d8a4405d3 None admin] PATH_INFO: `/v3/domains/f67184c979d84a80aab0131c0e8417b7` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:56.777096 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b42366-e365-4415-b576-e59d8a4405d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f67184c979d84a80aab0131c0e8417b7)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:56.779729 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b42366-e365-4415-b576-e59d8a4405d3 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:56.782919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 581/1176] 10.222.0.88 () {56 vars in 1183 bytes} [Tue Aug 23 22:11:56 2022] GET /identity/v3/domains/f67184c979d84a80aab0131c0e8417b7 => generated 267 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:56.789277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1158f77-5ec9-4bfc-93f6-76de8e0e208c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:56.908127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1158f77-5ec9-4bfc-93f6-76de8e0e208c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:56.909969 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:56.955413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1ac97-4782-44a2-add5-b82176ab0eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:56.956116 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1ac97-4782-44a2-add5-b82176ab0eb6 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:56.956442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1ac97-4782-44a2-add5-b82176ab0eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:56.956762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1ac97-4782-44a2-add5-b82176ab0eb6 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:56.957532 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1ac97-4782-44a2-add5-b82176ab0eb6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:56.959078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1ac97-4782-44a2-add5-b82176ab0eb6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:56.990653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 596/1177] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:11:56 2022] POST /identity/v3/users => generated 311 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:59.143658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b8851-200d-4b5d-bea8-4bf71eb2586f None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:59.144220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b8851-200d-4b5d-bea8-4bf71eb2586f None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:59.144220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b8851-200d-4b5d-bea8-4bf71eb2586f None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:59.145534 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 582/1178] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:59.152401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0162d72-c062-4ed3-80a8-f02ef67a1a85 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:59.152943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0162d72-c062-4ed3-80a8-f02ef67a1a85 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:59.152943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0162d72-c062-4ed3-80a8-f02ef67a1a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:59.181295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a0162d72-c062-4ed3-80a8-f02ef67a1a85 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:59.182769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a0162d72-c062-4ed3-80a8-f02ef67a1a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:59.272093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 597/1179] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:59.548099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b8851-200d-4b5d-bea8-4bf71eb2586f None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:59.650295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:59.690080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b8851-200d-4b5d-bea8-4bf71eb2586f None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:59.729099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5bf56-953b-4662-ba39-ae24ec64e1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:59.730119 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5bf56-953b-4662-ba39-ae24ec64e1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:59.730471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5bf56-953b-4662-ba39-ae24ec64e1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:59.730820 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5bf56-953b-4662-ba39-ae24ec64e1a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:59.745617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5bf56-953b-4662-ba39-ae24ec64e1a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:59.750816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-52f5bf56-953b-4662-ba39-ae24ec64e1a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:59.750816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:59.750816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:59.750816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:59.750816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:59.750816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:59.750816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:59.750816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:59.750816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:59.750816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:59.750816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:59.750816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:59.750816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:59.750816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:59.750816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:59.750816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:59.752766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:59.752766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:59.752766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:59.752766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:59.752766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:59.752766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:59.752766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:59.752766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:59.752766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:59.752766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:59.752766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:59.752766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:59.752766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:59.752766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:59.752766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:59.752766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:59.754857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:59.754857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:59.754857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:59.754857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:59.754857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:59.754857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:59.754857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:59.754857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:59.754857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:59.754857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:59.754857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:59.754857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:11:59.754857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 583/1180] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:11:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:59.761129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0162d72-c062-4ed3-80a8-f02ef67a1a85 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:59.892187 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:59.915044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0162d72-c062-4ed3-80a8-f02ef67a1a85 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:59.971854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413498a7-c726-40ef-ae20-d55bef4010ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:59.972481 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413498a7-c726-40ef-ae20-d55bef4010ab None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:59.972783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413498a7-c726-40ef-ae20-d55bef4010ab None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:59.973038 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413498a7-c726-40ef-ae20-d55bef4010ab None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:59.973753 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413498a7-c726-40ef-ae20-d55bef4010ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:59.974055 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413498a7-c726-40ef-ae20-d55bef4010ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:59.975474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413498a7-c726-40ef-ae20-d55bef4010ab None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:59.985573 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 598/1181] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:11:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:59.993885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5bf56-953b-4662-ba39-ae24ec64e1a4 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:00.137782 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5bf56-953b-4662-ba39-ae24ec64e1a4 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:00.176243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:00.186189 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3272cfc-3a16-4c34-9a85-fa62ffa31ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:00.186841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3272cfc-3a16-4c34-9a85-fa62ffa31ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:00.187116 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3272cfc-3a16-4c34-9a85-fa62ffa31ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:00.187389 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3272cfc-3a16-4c34-9a85-fa62ffa31ab8 None admin] PATH_INFO: `/v3/users/8fb2dac966c3453d94b49d0507c62122` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:00.205027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3272cfc-3a16-4c34-9a85-fa62ffa31ab8 None admin] RBAC: Authorizing `identity:get_user(user_id=8fb2dac966c3453d94b49d0507c62122)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:00.206555 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3272cfc-3a16-4c34-9a85-fa62ffa31ab8 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:00.208313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 584/1182] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:59 2022] GET /identity/v3/users/8fb2dac966c3453d94b49d0507c62122 => generated 311 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:12:00.215352 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413498a7-c726-40ef-ae20-d55bef4010ab None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:00.376582 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413498a7-c726-40ef-ae20-d55bef4010ab None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:00.429896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3f2c2-b8f8-4e37-b1cb-5cca2c839836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:00.430409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3f2c2-b8f8-4e37-b1cb-5cca2c839836 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:00.430704 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3f2c2-b8f8-4e37-b1cb-5cca2c839836 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:00.430972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3f2c2-b8f8-4e37-b1cb-5cca2c839836 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:00.471501 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:00.541550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3f2c2-b8f8-4e37-b1cb-5cca2c839836 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:00.543763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3f2c2-b8f8-4e37-b1cb-5cca2c839836 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:00.715242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 599/1183] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:12:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:12:00.717469 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3272cfc-3a16-4c34-9a85-fa62ffa31ab8 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:00.919292 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3272cfc-3a16-4c34-9a85-fa62ffa31ab8 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:00.967883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac35c24-5875-4212-be14-c2a723f80927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:00.968581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac35c24-5875-4212-be14-c2a723f80927 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:00.968950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac35c24-5875-4212-be14-c2a723f80927 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:00.969333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac35c24-5875-4212-be14-c2a723f80927 None admin] PATH_INFO: `/v3/projects/c0f605658a68476caf5006fa4733e1cf` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:00.970919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac35c24-5875-4212-be14-c2a723f80927 None admin] RBAC: Authorizing `identity:get_project(project_id=c0f605658a68476caf5006fa4733e1cf)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:00.972490 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac35c24-5875-4212-be14-c2a723f80927 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:00.974386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 585/1184] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:12:00 2022] GET /identity/v3/projects/c0f605658a68476caf5006fa4733e1cf => generated 361 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:00.985717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3f2c2-b8f8-4e37-b1cb-5cca2c839836 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:01.048004 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:01.113573 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3f2c2-b8f8-4e37-b1cb-5cca2c839836 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:01.160619 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3401592-85e0-4082-a403-383e8682c284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:01.161678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3401592-85e0-4082-a403-383e8682c284 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:01.162167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3401592-85e0-4082-a403-383e8682c284 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:01.162649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3401592-85e0-4082-a403-383e8682c284 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:01.164003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3401592-85e0-4082-a403-383e8682c284 None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=c0f605658a68476caf5006fa4733e1cf, user.id=8fb2dac966c3453d94b49d0507c62122) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:12:01.164536 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3401592-85e0-4082-a403-383e8682c284 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:01.168082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3401592-85e0-4082-a403-383e8682c284 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:01.178506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 600/1185] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:12:00 2022] GET /identity/v3/role_assignments?user.id=8fb2dac966c3453d94b49d0507c62122&scope.project.id=c0f605658a68476caf5006fa4733e1cf&role.id=dc5d71ab685e420697f520aa3a368177 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:12:03.578657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4c8c1-25bf-4c69-8bda-bb1ccfcba179 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:03.579511 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4c8c1-25bf-4c69-8bda-bb1ccfcba179 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:03.579511 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4c8c1-25bf-4c69-8bda-bb1ccfcba179 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:03.581045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 586/1186] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:12:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:12:03.588016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752dba3-99c7-401d-bac1-48fc8a462633 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:03.589393 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752dba3-99c7-401d-bac1-48fc8a462633 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:03.589636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752dba3-99c7-401d-bac1-48fc8a462633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:03.621419 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-5752dba3-99c7-401d-bac1-48fc8a462633 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:12:03.622661 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5752dba3-99c7-401d-bac1-48fc8a462633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:12:03.700132 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 601/1187] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:12:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:12:03.983305 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4c8c1-25bf-4c69-8bda-bb1ccfcba179 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:04.091679 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:04.152181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4c8c1-25bf-4c69-8bda-bb1ccfcba179 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:04.198129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859722c5-56b6-4dc6-b6e2-4f57560546c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:04.198925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859722c5-56b6-4dc6-b6e2-4f57560546c7 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:04.199305 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859722c5-56b6-4dc6-b6e2-4f57560546c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:04.199707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859722c5-56b6-4dc6-b6e2-4f57560546c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:04.213118 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859722c5-56b6-4dc6-b6e2-4f57560546c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:12:04.218401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-859722c5-56b6-4dc6-b6e2-4f57560546c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:12:04.218401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:12:04.218401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:12:04.218401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:12:04.218401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:12:04.218401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:12:04.218401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:12:04.218401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:12:04.218401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:12:04.218401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:12:04.218401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:12:04.218401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:12:04.218401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:12:04.218401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:12:04.218401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:12:04.218401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:12:04.220480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:12:04.220480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:12:04.220480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:12:04.220480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:12:04.220480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:12:04.220480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:12:04.220480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:12:04.220480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:12:04.220480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:12:04.220480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:12:04.220480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:12:04.220480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:12:04.220480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:12:04.220480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:12:04.220480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:12:04.220480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:12:04.222156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:12:04.222156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:12:04.222156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:12:04.222156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:12:04.222156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:12:04.222156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:12:04.222156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:12:04.222156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:12:04.222156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:12:04.222156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:12:04.222156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:12:04.222156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:12:04.222156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 587/1188] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:12:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:12:04.230012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752dba3-99c7-401d-bac1-48fc8a462633 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:04.318246 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:04.382038 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752dba3-99c7-401d-bac1-48fc8a462633 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:04.424716 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ee395-bf5b-4ba1-9139-2d22ddea7a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:04.425454 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ee395-bf5b-4ba1-9139-2d22ddea7a5a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:04.425781 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ee395-bf5b-4ba1-9139-2d22ddea7a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:04.426182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ee395-bf5b-4ba1-9139-2d22ddea7a5a None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:04.426993 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ee395-bf5b-4ba1-9139-2d22ddea7a5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:12:04.427341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ee395-bf5b-4ba1-9139-2d22ddea7a5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:04.428849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ee395-bf5b-4ba1-9139-2d22ddea7a5a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:04.439848 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 602/1189] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:12:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:04.451058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859722c5-56b6-4dc6-b6e2-4f57560546c7 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:04.594281 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859722c5-56b6-4dc6-b6e2-4f57560546c7 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:04.611407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:04.637175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30276ded-16a7-4327-9be4-5c8f4a1a491c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:04.638104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30276ded-16a7-4327-9be4-5c8f4a1a491c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:04.638515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30276ded-16a7-4327-9be4-5c8f4a1a491c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:04.639025 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30276ded-16a7-4327-9be4-5c8f4a1a491c None admin] PATH_INFO: `/v3/users/8fb2dac966c3453d94b49d0507c62122` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:04.640676 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30276ded-16a7-4327-9be4-5c8f4a1a491c None admin] RBAC: Authorizing `identity:get_user(user_id=8fb2dac966c3453d94b49d0507c62122)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:04.642346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30276ded-16a7-4327-9be4-5c8f4a1a491c None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:04.644176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 588/1190] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:12:04 2022] GET /identity/v3/users/8fb2dac966c3453d94b49d0507c62122 => generated 311 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:04.654223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ee395-bf5b-4ba1-9139-2d22ddea7a5a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:04.827287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ee395-bf5b-4ba1-9139-2d22ddea7a5a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:04.836159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:04.869461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466bf68-e92d-48ea-9f5b-7d031ccf1215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:04.870073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466bf68-e92d-48ea-9f5b-7d031ccf1215 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:04.870387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466bf68-e92d-48ea-9f5b-7d031ccf1215 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:04.870707 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466bf68-e92d-48ea-9f5b-7d031ccf1215 None admin] PATH_INFO: `/v3/projects/c0f605658a68476caf5006fa4733e1cf` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:04.871956 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466bf68-e92d-48ea-9f5b-7d031ccf1215 None admin] RBAC: Authorizing `identity:get_project(project_id=c0f605658a68476caf5006fa4733e1cf)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:04.873635 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466bf68-e92d-48ea-9f5b-7d031ccf1215 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:04.875412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 603/1191] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:12:04 2022] GET /identity/v3/projects/c0f605658a68476caf5006fa4733e1cf => generated 361 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:04.884516 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30276ded-16a7-4327-9be4-5c8f4a1a491c None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:05.066729 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30276ded-16a7-4327-9be4-5c8f4a1a491c None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:05.133073 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e84fe8-7056-4cde-a84c-b93f344252b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:05.134173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e84fe8-7056-4cde-a84c-b93f344252b7 None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:05.134679 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e84fe8-7056-4cde-a84c-b93f344252b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:05.135215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e84fe8-7056-4cde-a84c-b93f344252b7 None admin] PATH_INFO: `/v3/projects/c0f605658a68476caf5006fa4733e1cf/users/8fb2dac966c3453d94b49d0507c62122/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:05.138759 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e84fe8-7056-4cde-a84c-b93f344252b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0f605658a68476caf5006fa4733e1cf, user_id=8fb2dac966c3453d94b49d0507c62122, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:05.141573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e84fe8-7056-4cde-a84c-b93f344252b7 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:05.164115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 589/1192] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:12:04 2022] PUT /identity/v3/projects/c0f605658a68476caf5006fa4733e1cf/users/8fb2dac966c3453d94b49d0507c62122/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:12:07.329726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3359af9-16aa-4721-b8da-ec8245c79c09 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:07.330547 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3359af9-16aa-4721-b8da-ec8245c79c09 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:07.330765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3359af9-16aa-4721-b8da-ec8245c79c09 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:07.332200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 604/1193] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:12:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:12:07.339272 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba93141-6729-4152-8d1d-30a0a1caaa7e None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:07.340350 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba93141-6729-4152-8d1d-30a0a1caaa7e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:07.340740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba93141-6729-4152-8d1d-30a0a1caaa7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:07.372193 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-1ba93141-6729-4152-8d1d-30a0a1caaa7e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:12:07.374433 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-1ba93141-6729-4152-8d1d-30a0a1caaa7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:12:07.442016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:07.495364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 590/1194] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:12:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:12:07.947049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3359af9-16aa-4721-b8da-ec8245c79c09 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:08.113005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3359af9-16aa-4721-b8da-ec8245c79c09 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:08.157312 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa8e1a-4fda-46a3-80d8-22bb04398e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:08.158018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa8e1a-4fda-46a3-80d8-22bb04398e42 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:08.158337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa8e1a-4fda-46a3-80d8-22bb04398e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:08.158645 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa8e1a-4fda-46a3-80d8-22bb04398e42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:08.171650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa8e1a-4fda-46a3-80d8-22bb04398e42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:12:08.176728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-b2fa8e1a-4fda-46a3-80d8-22bb04398e42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:12:08.176728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:12:08.176728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:12:08.176728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:12:08.176728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:12:08.176728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:12:08.176728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:12:08.176728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:12:08.176728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:12:08.176728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:12:08.176728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:12:08.176728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:12:08.176728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:12:08.176728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:12:08.176728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:12:08.176728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:12:08.178850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:12:08.178850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:12:08.178850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:12:08.178850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:12:08.178850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:12:08.178850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:12:08.178850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:12:08.178850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:12:08.178850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:12:08.178850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:12:08.178850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:12:08.178850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:12:08.178850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:12:08.178850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:12:08.178850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:12:08.178850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:12:08.180893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:12:08.180893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:12:08.180893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:12:08.180893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:12:08.180893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:12:08.180893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:12:08.180893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:12:08.180893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:12:08.180893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:12:08.180893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:12:08.180893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:12:08.180893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:12:08.180893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 605/1195] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:12:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:12:08.186358 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba93141-6729-4152-8d1d-30a0a1caaa7e None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:08.329471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba93141-6729-4152-8d1d-30a0a1caaa7e None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:08.381941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e8bb9-7254-40a8-9ff0-a87607f2c765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:08.382977 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e8bb9-7254-40a8-9ff0-a87607f2c765 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:08.382977 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e8bb9-7254-40a8-9ff0-a87607f2c765 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:08.383273 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e8bb9-7254-40a8-9ff0-a87607f2c765 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:08.383968 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e8bb9-7254-40a8-9ff0-a87607f2c765 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:12:08.384315 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e8bb9-7254-40a8-9ff0-a87607f2c765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:08.386531 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e8bb9-7254-40a8-9ff0-a87607f2c765 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:08.396455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 591/1196] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:12:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:12:08.407113 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa8e1a-4fda-46a3-80d8-22bb04398e42 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:08.450465 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:08.562133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa8e1a-4fda-46a3-80d8-22bb04398e42 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:08.636987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1562f-e1ab-48e7-88ab-c7a8c6673589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:08.638032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1562f-e1ab-48e7-88ab-c7a8c6673589 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:08.638517 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1562f-e1ab-48e7-88ab-c7a8c6673589 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:08.638991 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1562f-e1ab-48e7-88ab-c7a8c6673589 None admin] PATH_INFO: `/v3/users/8fb2dac966c3453d94b49d0507c62122` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:08.641104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1562f-e1ab-48e7-88ab-c7a8c6673589 None admin] RBAC: Authorizing `identity:get_user(user_id=8fb2dac966c3453d94b49d0507c62122)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:08.643873 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1562f-e1ab-48e7-88ab-c7a8c6673589 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:08.647320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 606/1197] 10.222.0.88 () {56 vars in 1177 bytes} [Tue Aug 23 22:12:08 2022] GET /identity/v3/users/8fb2dac966c3453d94b49d0507c62122 => generated 311 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:08.654475 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e8bb9-7254-40a8-9ff0-a87607f2c765 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:08.695370 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:08.780541 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e8bb9-7254-40a8-9ff0-a87607f2c765 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:08.822020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63188a09-0c72-4576-9aed-2b3a81b4c835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:08.822759 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63188a09-0c72-4576-9aed-2b3a81b4c835 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:08.823074 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63188a09-0c72-4576-9aed-2b3a81b4c835 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:08.823384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63188a09-0c72-4576-9aed-2b3a81b4c835 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:08.911350 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63188a09-0c72-4576-9aed-2b3a81b4c835 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:08.912890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63188a09-0c72-4576-9aed-2b3a81b4c835 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:08.956344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:09.049417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 592/1198] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:12:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:12:09.059471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1562f-e1ab-48e7-88ab-c7a8c6673589 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:09.179434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1562f-e1ab-48e7-88ab-c7a8c6673589 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:09.182128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:09.222876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac4ff7-2a40-4839-8091-d94ad2577806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:09.223732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac4ff7-2a40-4839-8091-d94ad2577806 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:09.224105 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac4ff7-2a40-4839-8091-d94ad2577806 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:09.224478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac4ff7-2a40-4839-8091-d94ad2577806 None admin] PATH_INFO: `/v3/projects/c0f605658a68476caf5006fa4733e1cf` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:09.225795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac4ff7-2a40-4839-8091-d94ad2577806 None admin] RBAC: Authorizing `identity:get_project(project_id=c0f605658a68476caf5006fa4733e1cf)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:09.227337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac4ff7-2a40-4839-8091-d94ad2577806 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:09.230045 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 607/1199] 10.222.0.88 () {56 vars in 1186 bytes} [Tue Aug 23 22:12:09 2022] GET /identity/v3/projects/c0f605658a68476caf5006fa4733e1cf => generated 361 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:09.236581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63188a09-0c72-4576-9aed-2b3a81b4c835 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:09.355728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63188a09-0c72-4576-9aed-2b3a81b4c835 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:09.409637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc342a45-4e8f-489c-a498-1167de8eb4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:09.410704 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc342a45-4e8f-489c-a498-1167de8eb4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:09.410704 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc342a45-4e8f-489c-a498-1167de8eb4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:09.411012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc342a45-4e8f-489c-a498-1167de8eb4f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:09.411705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc342a45-4e8f-489c-a498-1167de8eb4f4 None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=c0f605658a68476caf5006fa4733e1cf, user.id=8fb2dac966c3453d94b49d0507c62122) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:12:09.412051 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc342a45-4e8f-489c-a498-1167de8eb4f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:09.413649 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc342a45-4e8f-489c-a498-1167de8eb4f4 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:09.415929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:09.422781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 593/1200] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:12:09 2022] GET /identity/v3/role_assignments?user.id=8fb2dac966c3453d94b49d0507c62122&scope.project.id=c0f605658a68476caf5006fa4733e1cf&role.id=dc5d71ab685e420697f520aa3a368177 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:12:19.218149 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=15 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:19.421556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:23.535599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fec755-01d9-4a1a-896d-bc012681a8c8 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:23.537123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fec755-01d9-4a1a-896d-bc012681a8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:23.537123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fec755-01d9-4a1a-896d-bc012681a8c8 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:23.543178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 608/1201] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:13:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:13:23.553618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5509e-7da9-4576-b6f2-14dc62c5f355 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:23.554954 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5509e-7da9-4576-b6f2-14dc62c5f355 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:23.555133 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5509e-7da9-4576-b6f2-14dc62c5f355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:23.591524 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-71e5509e-7da9-4576-b6f2-14dc62c5f355 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:13:23.594071 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-71e5509e-7da9-4576-b6f2-14dc62c5f355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:13:23.749117 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 594/1202] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:13:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:13:24.302923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fec755-01d9-4a1a-896d-bc012681a8c8 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:24.547518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fec755-01d9-4a1a-896d-bc012681a8c8 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:24.619030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eed29d-e657-478b-ba91-20f25baed5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:24.620343 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eed29d-e657-478b-ba91-20f25baed5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:24.620843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eed29d-e657-478b-ba91-20f25baed5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:24.621327 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eed29d-e657-478b-ba91-20f25baed5c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:24.641654 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eed29d-e657-478b-ba91-20f25baed5c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:24.643539 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eed29d-e657-478b-ba91-20f25baed5c6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:24.672824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-92eed29d-e657-478b-ba91-20f25baed5c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:24.672824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:24.672824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:24.672824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:24.672824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:24.672824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:24.672824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:24.672824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:24.672824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:24.672824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:24.672824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:24.672824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:24.672824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:13:24.672824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:13:24.672824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:13:24.672824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:24.675693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:13:24.679658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:13:24.679658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:24.679658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:13:24.686295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 609/1203] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:24.686677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5509e-7da9-4576-b6f2-14dc62c5f355 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:24.925884 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5509e-7da9-4576-b6f2-14dc62c5f355 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:24.992686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:25.003076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01411e1-8b3b-4fda-99b7-8e28899289fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:25.004388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01411e1-8b3b-4fda-99b7-8e28899289fc None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:25.005125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01411e1-8b3b-4fda-99b7-8e28899289fc None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:25.005970 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01411e1-8b3b-4fda-99b7-8e28899289fc None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:25.007881 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01411e1-8b3b-4fda-99b7-8e28899289fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:25.008475 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01411e1-8b3b-4fda-99b7-8e28899289fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:25.011208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01411e1-8b3b-4fda-99b7-8e28899289fc None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:25.038388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 595/1204] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:25.049663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eed29d-e657-478b-ba91-20f25baed5c6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:25.160101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:25.276769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eed29d-e657-478b-ba91-20f25baed5c6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:25.354205 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75046da0-ee12-44a2-bac7-3e871c0e7c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:25.355657 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75046da0-ee12-44a2-bac7-3e871c0e7c09 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:25.357892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75046da0-ee12-44a2-bac7-3e871c0e7c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:25.358420 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75046da0-ee12-44a2-bac7-3e871c0e7c09 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:25.364055 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75046da0-ee12-44a2-bac7-3e871c0e7c09 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:25.368959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75046da0-ee12-44a2-bac7-3e871c0e7c09 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:25.417112 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 610/1205] 10.222.0.88 () {60 vars in 1132 bytes} [Tue Aug 23 22:13:25 2022] POST /identity/v3/users => generated 251 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:13:28.586052 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06dd225-f598-4cbf-b972-6033ee53e46c None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:28.586052 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06dd225-f598-4cbf-b972-6033ee53e46c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:28.587086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06dd225-f598-4cbf-b972-6033ee53e46c None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:28.589218 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 596/1206] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:13:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:13:28.599633 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32aaf45-c577-473a-a75e-8dff24ce8b21 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:28.600547 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32aaf45-c577-473a-a75e-8dff24ce8b21 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:28.600867 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32aaf45-c577-473a-a75e-8dff24ce8b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:28.649315 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b32aaf45-c577-473a-a75e-8dff24ce8b21 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:13:28.652585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b32aaf45-c577-473a-a75e-8dff24ce8b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:13:28.847542 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 611/1207] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:13:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:13:29.255067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06dd225-f598-4cbf-b972-6033ee53e46c None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:29.459325 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06dd225-f598-4cbf-b972-6033ee53e46c None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:29.517809 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300ae7a-6024-4d21-9410-5ff0424473ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:29.518603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300ae7a-6024-4d21-9410-5ff0424473ec None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:29.519032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300ae7a-6024-4d21-9410-5ff0424473ec None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:29.519479 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300ae7a-6024-4d21-9410-5ff0424473ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:29.543003 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300ae7a-6024-4d21-9410-5ff0424473ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:13:29.554520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-a300ae7a-6024-4d21-9410-5ff0424473ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:13:29.554520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:29.554520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:29.554520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:29.554520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:29.554520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:29.554520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:29.554520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:29.554520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:29.554520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:29.554520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:29.554520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:29.554520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:13:29.554520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:13:29.554520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:13:29.554520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:13:29.556705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:13:29.556705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:13:29.556705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:13:29.556705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:13:29.556705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:29.556705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:29.556705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:29.556705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:29.556705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:29.556705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:29.556705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:29.556705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:29.556705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:29.556705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:29.556705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:29.556705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:29.558994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:29.558994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:29.558994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:29.558994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:29.558994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:13:29.558994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:13:29.558994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:13:29.558994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:13:29.558994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:13:29.558994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:13:29.558994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:13:29.558994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:13:29.558994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 597/1208] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:13:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:29.565520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32aaf45-c577-473a-a75e-8dff24ce8b21 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:29.663497 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:29.775043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32aaf45-c577-473a-a75e-8dff24ce8b21 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:29.854563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24cd0d-8f6d-4840-844a-191f3ab56fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:29.856529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24cd0d-8f6d-4840-844a-191f3ab56fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:29.857568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24cd0d-8f6d-4840-844a-191f3ab56fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:29.858733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24cd0d-8f6d-4840-844a-191f3ab56fa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:29.861212 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24cd0d-8f6d-4840-844a-191f3ab56fa4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:29.862200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24cd0d-8f6d-4840-844a-191f3ab56fa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:29.864447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24cd0d-8f6d-4840-844a-191f3ab56fa4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:29.889247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 612/1209] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:13:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:13:29.905297 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300ae7a-6024-4d21-9410-5ff0424473ec None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:30.004500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:30.112252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300ae7a-6024-4d21-9410-5ff0424473ec None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:30.194583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a72110-ba13-4241-a19a-552f00424efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:30.195742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a72110-ba13-4241-a19a-552f00424efa None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:30.196588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a72110-ba13-4241-a19a-552f00424efa None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:30.196787 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a72110-ba13-4241-a19a-552f00424efa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:30.218088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a72110-ba13-4241-a19a-552f00424efa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:30.220453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a72110-ba13-4241-a19a-552f00424efa None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:30.254081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-d7a72110-ba13-4241-a19a-552f00424efa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:30.254081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:30.254081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:30.254081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:30.254081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:30.254081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:30.254081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:30.254081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:30.254081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:30.254081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:30.254081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:30.254081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:30.254081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:13:30.254081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:13:30.254081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:13:30.254081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:30.256696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:13:30.259386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:13:30.259386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:30.259386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:13:30.259386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 598/1210] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:30.274201 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24cd0d-8f6d-4840-844a-191f3ab56fa4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:30.499254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:30.504087 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24cd0d-8f6d-4840-844a-191f3ab56fa4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:30.581296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1db515-7e71-49b3-acd5-cb0dd5eb3a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:30.581892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1db515-7e71-49b3-acd5-cb0dd5eb3a84 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:30.582375 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1db515-7e71-49b3-acd5-cb0dd5eb3a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:30.582558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1db515-7e71-49b3-acd5-cb0dd5eb3a84 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:30.583218 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1db515-7e71-49b3-acd5-cb0dd5eb3a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:30.583600 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1db515-7e71-49b3-acd5-cb0dd5eb3a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:30.586601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1db515-7e71-49b3-acd5-cb0dd5eb3a84 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:30.613185 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 613/1211] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:30.623762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a72110-ba13-4241-a19a-552f00424efa None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:30.846169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a72110-ba13-4241-a19a-552f00424efa None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:30.849861 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:30.926114 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8b07e-13a5-46e4-bc92-8fe989865941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:30.927230 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8b07e-13a5-46e4-bc92-8fe989865941 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:30.927641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8b07e-13a5-46e4-bc92-8fe989865941 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:30.928128 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8b07e-13a5-46e4-bc92-8fe989865941 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:30.948800 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8b07e-13a5-46e4-bc92-8fe989865941 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:30.951655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8b07e-13a5-46e4-bc92-8fe989865941 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:30.974718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-c2f8b07e-13a5-46e4-bc92-8fe989865941 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:13:30.974718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:30.974718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:30.974718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:30.974718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:30.974718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:30.974718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:30.974718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:30.974718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:30.974718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:30.974718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:30.974718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:30.974718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:13:30.974718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:13:30.974718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:13:30.974718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:30.977090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:30.978844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:30.978844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:30.978844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:13:30.978844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:13:30.978844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:13:30.978844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:13:30.978844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:13:30.978844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:13:30.978844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:13:30.978844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:13:30.985030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 599/1212] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:30.986650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1db515-7e71-49b3-acd5-cb0dd5eb3a84 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:31.208422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1db515-7e71-49b3-acd5-cb0dd5eb3a84 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:31.309099 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45393368-8693-4b1c-bfd5-61de1862d406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:31.310061 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45393368-8693-4b1c-bfd5-61de1862d406 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:31.310449 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45393368-8693-4b1c-bfd5-61de1862d406 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:31.310923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45393368-8693-4b1c-bfd5-61de1862d406 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:31.312030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45393368-8693-4b1c-bfd5-61de1862d406 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:31.312572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45393368-8693-4b1c-bfd5-61de1862d406 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:31.315375 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45393368-8693-4b1c-bfd5-61de1862d406 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:31.357628 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 614/1213] 10.222.0.88 () {56 vars in 1143 bytes} [Tue Aug 23 22:13:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:31.368572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8b07e-13a5-46e4-bc92-8fe989865941 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:31.601257 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8b07e-13a5-46e4-bc92-8fe989865941 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:31.671904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62883f-2323-4303-9cd3-9a63a3f14c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:31.673355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62883f-2323-4303-9cd3-9a63a3f14c86 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:31.673936 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62883f-2323-4303-9cd3-9a63a3f14c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:31.674569 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62883f-2323-4303-9cd3-9a63a3f14c86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:31.749162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:31.820181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62883f-2323-4303-9cd3-9a63a3f14c86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:31.822395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62883f-2323-4303-9cd3-9a63a3f14c86 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:32.055912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 600/1214] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:13:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:13:32.063627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45393368-8693-4b1c-bfd5-61de1862d406 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:32.096904 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:32.287742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45393368-8693-4b1c-bfd5-61de1862d406 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:32.368756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff3c89-ce8f-4c59-b667-9936725ce3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:32.369622 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff3c89-ce8f-4c59-b667-9936725ce3ba None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:32.370144 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff3c89-ce8f-4c59-b667-9936725ce3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:32.370340 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff3c89-ce8f-4c59-b667-9936725ce3ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:32.392491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff3c89-ce8f-4c59-b667-9936725ce3ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:32.395568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff3c89-ce8f-4c59-b667-9936725ce3ba None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:32.423943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-29ff3c89-ce8f-4c59-b667-9936725ce3ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:32.423943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:32.423943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:32.423943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:32.423943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:32.423943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:32.423943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:32.423943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:32.423943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:32.423943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:32.423943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:32.423943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:32.423943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:13:32.423943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:13:32.423943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:13:32.423943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:32.426432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:13:32.429734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:13:32.429734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:32.429734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:13:32.433455 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 615/1215] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:32.435498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62883f-2323-4303-9cd3-9a63a3f14c86 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:32.548393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:32.643854 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62883f-2323-4303-9cd3-9a63a3f14c86 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:32.733499 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c33099-757c-45cf-845e-49b038ec86d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:32.734809 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c33099-757c-45cf-845e-49b038ec86d2 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:32.735446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c33099-757c-45cf-845e-49b038ec86d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:32.736119 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c33099-757c-45cf-845e-49b038ec86d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:32.737610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c33099-757c-45cf-845e-49b038ec86d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:32.738273 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c33099-757c-45cf-845e-49b038ec86d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:32.744841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c33099-757c-45cf-845e-49b038ec86d2 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:32.778203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 601/1216] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:32.780817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff3c89-ce8f-4c59-b667-9936725ce3ba None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:32.965520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:33.029741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff3c89-ce8f-4c59-b667-9936725ce3ba None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:33.104665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63806c2e-5830-42e6-a78b-49876c3fb2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:33.105715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63806c2e-5830-42e6-a78b-49876c3fb2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:33.106163 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63806c2e-5830-42e6-a78b-49876c3fb2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:33.106670 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63806c2e-5830-42e6-a78b-49876c3fb2d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:33.120020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63806c2e-5830-42e6-a78b-49876c3fb2d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:33.122700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63806c2e-5830-42e6-a78b-49876c3fb2d0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:33.146464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-63806c2e-5830-42e6-a78b-49876c3fb2d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:13:33.146464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:33.146464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:33.146464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:33.146464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:33.146464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:33.146464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:33.146464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:33.146464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:33.146464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:33.146464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:33.146464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:33.146464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:13:33.146464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:13:33.146464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:33.148959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:33.150504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:13:33.150504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:13:33.150504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:13:33.150504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:13:33.150504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:13:33.150504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:13:33.150504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:13:33.150504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:13:33.152894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 616/1217] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:13:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:13:33.170282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c33099-757c-45cf-845e-49b038ec86d2 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:33.407386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c33099-757c-45cf-845e-49b038ec86d2 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:33.472104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:33.502080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636dc1c-6452-4e4e-bed0-0860e220e2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:33.503363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636dc1c-6452-4e4e-bed0-0860e220e2de None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:33.503988 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636dc1c-6452-4e4e-bed0-0860e220e2de None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:33.504594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636dc1c-6452-4e4e-bed0-0860e220e2de None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:33.506080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636dc1c-6452-4e4e-bed0-0860e220e2de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:33.506795 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636dc1c-6452-4e4e-bed0-0860e220e2de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:33.509845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636dc1c-6452-4e4e-bed0-0860e220e2de None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:33.536402 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 602/1218] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:13:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:33.547235 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63806c2e-5830-42e6-a78b-49876c3fb2d0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:33.804291 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63806c2e-5830-42e6-a78b-49876c3fb2d0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:33.855584 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:33.864209 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27167d22-f4c1-42c5-8483-a0fc7e40ec51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:33.864828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27167d22-f4c1-42c5-8483-a0fc7e40ec51 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:33.865129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27167d22-f4c1-42c5-8483-a0fc7e40ec51 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:33.865426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27167d22-f4c1-42c5-8483-a0fc7e40ec51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:33.866330 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27167d22-f4c1-42c5-8483-a0fc7e40ec51 None admin] RBAC: Adding query filter params (role.id=23ea214daa734346a6da3a0a8c50d23a, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=e642dd7a07ca4afaa40c2769299d8bad) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:33.866330 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27167d22-f4c1-42c5-8483-a0fc7e40ec51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:33.868091 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27167d22-f4c1-42c5-8483-a0fc7e40ec51 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:33.885111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 617/1219] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:13:33 2022] GET /identity/v3/role_assignments?user.id=e642dd7a07ca4afaa40c2769299d8bad&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=23ea214daa734346a6da3a0a8c50d23a => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:36.524164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4363e08-b440-4e6f-b11d-6d2ba108758b None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:36.524988 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4363e08-b440-4e6f-b11d-6d2ba108758b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:36.525285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4363e08-b440-4e6f-b11d-6d2ba108758b None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:36.527288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 603/1220] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:13:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:13:36.537171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c9cb9-8f8e-482a-bc4e-f7b95c442367 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:36.538845 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c9cb9-8f8e-482a-bc4e-f7b95c442367 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:36.538845 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c9cb9-8f8e-482a-bc4e-f7b95c442367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:36.572950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-af2c9cb9-8f8e-482a-bc4e-f7b95c442367 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:13:36.575717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-af2c9cb9-8f8e-482a-bc4e-f7b95c442367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:13:36.739073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 618/1221] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:13:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:13:37.155390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4363e08-b440-4e6f-b11d-6d2ba108758b None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:37.363515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4363e08-b440-4e6f-b11d-6d2ba108758b None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:37.421254 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee132e86-3490-4fa4-9a7a-ed0b3e9fcb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:37.422320 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee132e86-3490-4fa4-9a7a-ed0b3e9fcb48 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:37.426618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee132e86-3490-4fa4-9a7a-ed0b3e9fcb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:37.426618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee132e86-3490-4fa4-9a7a-ed0b3e9fcb48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:37.438938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee132e86-3490-4fa4-9a7a-ed0b3e9fcb48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:13:37.450904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-ee132e86-3490-4fa4-9a7a-ed0b3e9fcb48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:13:37.450904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:37.450904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:37.450904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:37.450904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:37.450904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:37.450904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:37.450904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:37.450904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:37.450904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:37.450904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:37.450904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:37.450904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:13:37.450904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:13:37.450904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:13:37.450904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:13:37.452219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:13:37.452219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:13:37.452219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:13:37.452219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:13:37.452219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:37.452219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:37.452219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:37.452219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:37.452219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:37.452219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:37.452219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:37.452219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:37.452219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:37.452219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:37.452219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:37.452219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:37.453588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:37.453588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:37.453588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:37.453588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:37.453588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:13:37.453588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:13:37.453588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:13:37.453588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:13:37.453588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:13:37.453588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:13:37.453588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:13:37.453588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:13:37.453588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 604/1222] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:13:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:37.461630 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c9cb9-8f8e-482a-bc4e-f7b95c442367 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:37.641488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c9cb9-8f8e-482a-bc4e-f7b95c442367 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:37.692491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:37.709723 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648be79-fcbb-4ac7-801d-dee06c22b977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:37.710473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648be79-fcbb-4ac7-801d-dee06c22b977 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:37.710799 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648be79-fcbb-4ac7-801d-dee06c22b977 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:37.711103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648be79-fcbb-4ac7-801d-dee06c22b977 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:37.711819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648be79-fcbb-4ac7-801d-dee06c22b977 None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:37.712251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648be79-fcbb-4ac7-801d-dee06c22b977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:37.713726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648be79-fcbb-4ac7-801d-dee06c22b977 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:37.724494 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 619/1223] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:13:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:37.732752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee132e86-3490-4fa4-9a7a-ed0b3e9fcb48 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:37.805705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:37.926328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee132e86-3490-4fa4-9a7a-ed0b3e9fcb48 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:37.985581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5182c6-bd8c-46fa-a497-278df5e29efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:37.986298 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5182c6-bd8c-46fa-a497-278df5e29efb None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:37.986744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5182c6-bd8c-46fa-a497-278df5e29efb None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:37.987142 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5182c6-bd8c-46fa-a497-278df5e29efb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:38.005610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5182c6-bd8c-46fa-a497-278df5e29efb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:38.008638 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5182c6-bd8c-46fa-a497-278df5e29efb None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:38.030576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-ea5182c6-bd8c-46fa-a497-278df5e29efb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:38.030576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:38.030576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:38.030576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:38.030576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:38.030576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:38.030576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:38.030576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:38.030576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:38.030576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:38.030576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:38.030576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:38.030576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:13:38.030576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:13:38.030576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:13:38.030576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:38.032839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:13:38.034848 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:13:38.034848 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:38.034848 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:13:38.034848 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 605/1224] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:38.041623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648be79-fcbb-4ac7-801d-dee06c22b977 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:38.232119 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648be79-fcbb-4ac7-801d-dee06c22b977 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:38.283624 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96731a7-9896-4ca7-97dd-b20b7d42e3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:38.285209 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96731a7-9896-4ca7-97dd-b20b7d42e3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:38.286500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96731a7-9896-4ca7-97dd-b20b7d42e3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:38.287400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96731a7-9896-4ca7-97dd-b20b7d42e3b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:38.289788 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96731a7-9896-4ca7-97dd-b20b7d42e3b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:38.290552 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96731a7-9896-4ca7-97dd-b20b7d42e3b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:38.296818 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96731a7-9896-4ca7-97dd-b20b7d42e3b7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:38.314257 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 620/1225] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:13:38.323575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5182c6-bd8c-46fa-a497-278df5e29efb None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:38.491088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:38.504784 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5182c6-bd8c-46fa-a497-278df5e29efb None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:38.593397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e907bf-0bcc-455c-8e57-a3c5132cf66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:38.594710 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e907bf-0bcc-455c-8e57-a3c5132cf66c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:38.595337 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e907bf-0bcc-455c-8e57-a3c5132cf66c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:38.595932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e907bf-0bcc-455c-8e57-a3c5132cf66c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:38.612372 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e907bf-0bcc-455c-8e57-a3c5132cf66c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:38.614847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e907bf-0bcc-455c-8e57-a3c5132cf66c None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:38.638683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-d4e907bf-0bcc-455c-8e57-a3c5132cf66c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:13:38.638683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:38.638683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:38.638683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:38.638683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:38.638683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:38.638683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:38.638683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:38.638683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:38.638683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:38.638683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:38.638683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:38.638683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:13:38.638683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:13:38.638683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:13:38.638683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:38.640808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:38.643055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:38.643055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:38.643055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:13:38.643055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:13:38.643055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:13:38.643055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:13:38.643055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:13:38.643055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:13:38.643055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:13:38.643055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:13:38.643055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 606/1226] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:13:38.650842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96731a7-9896-4ca7-97dd-b20b7d42e3b7 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:38.744806 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:38.864215 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96731a7-9896-4ca7-97dd-b20b7d42e3b7 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:38.939942 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559fd1e-04c3-402e-9033-1672a8b4c223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:38.940945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559fd1e-04c3-402e-9033-1672a8b4c223 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:38.941380 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559fd1e-04c3-402e-9033-1672a8b4c223 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:38.941706 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559fd1e-04c3-402e-9033-1672a8b4c223 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:38.942845 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559fd1e-04c3-402e-9033-1672a8b4c223 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:38.943289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559fd1e-04c3-402e-9033-1672a8b4c223 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:38.947807 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559fd1e-04c3-402e-9033-1672a8b4c223 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:38.983479 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 621/1227] 10.222.0.88 () {56 vars in 1143 bytes} [Tue Aug 23 22:13:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:38.993884 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e907bf-0bcc-455c-8e57-a3c5132cf66c None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:39.210471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e907bf-0bcc-455c-8e57-a3c5132cf66c None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:39.262847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf9ebf-9777-4ee2-b544-c38a82a5a91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:39.263952 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf9ebf-9777-4ee2-b544-c38a82a5a91e None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:39.264484 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf9ebf-9777-4ee2-b544-c38a82a5a91e None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:39.264976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf9ebf-9777-4ee2-b544-c38a82a5a91e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:39.288550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf9ebf-9777-4ee2-b544-c38a82a5a91e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:39.291705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf9ebf-9777-4ee2-b544-c38a82a5a91e None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:39.326665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-cecf9ebf-9777-4ee2-b544-c38a82a5a91e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:39.326665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:39.326665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:39.326665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:39.326665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:39.326665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:39.326665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:39.326665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:39.326665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:39.326665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:39.326665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:39.326665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:39.326665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:13:39.326665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:13:39.326665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:13:39.326665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:39.327866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:13:39.328932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:13:39.328932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:39.328932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:13:39.333383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 607/1228] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:39.339000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559fd1e-04c3-402e-9033-1672a8b4c223 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:39.583475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559fd1e-04c3-402e-9033-1672a8b4c223 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:39.587947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:39.659012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df874a-1060-4e00-8b4f-870124e2be1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:39.660360 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df874a-1060-4e00-8b4f-870124e2be1c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:39.660923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df874a-1060-4e00-8b4f-870124e2be1c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:39.661427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df874a-1060-4e00-8b4f-870124e2be1c None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:39.663996 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df874a-1060-4e00-8b4f-870124e2be1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:39.664706 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df874a-1060-4e00-8b4f-870124e2be1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:39.668617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df874a-1060-4e00-8b4f-870124e2be1c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:39.690670 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 622/1229] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:39.698958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf9ebf-9777-4ee2-b544-c38a82a5a91e None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:39.721099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:39.898793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf9ebf-9777-4ee2-b544-c38a82a5a91e None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:39.965408 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df927730-6cf1-4dc9-94c4-ab4e6a283bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:39.966526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df927730-6cf1-4dc9-94c4-ab4e6a283bdd None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:39.967008 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df927730-6cf1-4dc9-94c4-ab4e6a283bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:39.967508 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df927730-6cf1-4dc9-94c4-ab4e6a283bdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:39.985869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df927730-6cf1-4dc9-94c4-ab4e6a283bdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:39.987557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df927730-6cf1-4dc9-94c4-ab4e6a283bdd None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:40.007825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-df927730-6cf1-4dc9-94c4-ab4e6a283bdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:13:40.007825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:40.007825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:40.007825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:40.007825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:40.007825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:40.007825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:40.007825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:40.007825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:40.007825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:40.007825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:40.007825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:40.007825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:13:40.007825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:13:40.007825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:40.010896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:40.012643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:13:40.012643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:13:40.012643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:13:40.012643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:13:40.012643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:13:40.012643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:13:40.012643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:13:40.012643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:13:40.012643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 608/1230] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:13:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:13:40.018306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df874a-1060-4e00-8b4f-870124e2be1c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:40.172898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df874a-1060-4e00-8b4f-870124e2be1c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:40.229365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990af744-584b-4a9c-9c20-9a91fc2ecd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:40.230136 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990af744-584b-4a9c-9c20-9a91fc2ecd74 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:40.230472 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990af744-584b-4a9c-9c20-9a91fc2ecd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:40.230844 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990af744-584b-4a9c-9c20-9a91fc2ecd74 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:40.231815 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990af744-584b-4a9c-9c20-9a91fc2ecd74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:40.232100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990af744-584b-4a9c-9c20-9a91fc2ecd74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:40.233975 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990af744-584b-4a9c-9c20-9a91fc2ecd74 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:40.265761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 623/1231] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:13:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:13:40.274510 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df927730-6cf1-4dc9-94c4-ab4e6a283bdd None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:40.400263 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:40.453232 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df927730-6cf1-4dc9-94c4-ab4e6a283bdd None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:40.506293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0613a3-bceb-46b1-99e4-7118a1e5a2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:40.507120 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0613a3-bceb-46b1-99e4-7118a1e5a2fa None admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:40.507509 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0613a3-bceb-46b1-99e4-7118a1e5a2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:40.507970 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0613a3-bceb-46b1-99e4-7118a1e5a2fa None admin] PATH_INFO: `/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/e642dd7a07ca4afaa40c2769299d8bad/roles/23ea214daa734346a6da3a0a8c50d23a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:40.537976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0613a3-bceb-46b1-99e4-7118a1e5a2fa None admin] RBAC: Authorizing `identity:create_grant(project_id=8a7317af325f485e8fae49631cb5bc20, user_id=e642dd7a07ca4afaa40c2769299d8bad, role_id=23ea214daa734346a6da3a0a8c50d23a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:40.540221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0613a3-bceb-46b1-99e4-7118a1e5a2fa None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:40.569832 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 609/1232] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:13:40 2022] PUT /identity/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/e642dd7a07ca4afaa40c2769299d8bad/roles/23ea214daa734346a6da3a0a8c50d23a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:13:43.483601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33678b75-44f7-4b06-ae21-5fd6ca8661cb None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:43.484185 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33678b75-44f7-4b06-ae21-5fd6ca8661cb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:43.484561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33678b75-44f7-4b06-ae21-5fd6ca8661cb None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:43.487318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 624/1233] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:13:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:13:43.498836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93617a2c-1997-4f58-941b-1823ca0fabd1 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:43.499412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93617a2c-1997-4f58-941b-1823ca0fabd1 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:43.499768 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93617a2c-1997-4f58-941b-1823ca0fabd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:43.536243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-93617a2c-1997-4f58-941b-1823ca0fabd1 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:13:43.539154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-93617a2c-1997-4f58-941b-1823ca0fabd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:13:43.692279 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 610/1234] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:13:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:13:44.166112 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33678b75-44f7-4b06-ae21-5fd6ca8661cb None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:44.197273 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:44.322014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33678b75-44f7-4b06-ae21-5fd6ca8661cb None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:44.389152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb0500-b4bc-452b-9fa4-2ac056eebe2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:44.390115 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb0500-b4bc-452b-9fa4-2ac056eebe2f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:44.390492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb0500-b4bc-452b-9fa4-2ac056eebe2f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:44.390881 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb0500-b4bc-452b-9fa4-2ac056eebe2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:44.413258 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb0500-b4bc-452b-9fa4-2ac056eebe2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:13:44.420158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-e3cb0500-b4bc-452b-9fa4-2ac056eebe2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:13:44.420158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:44.420158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:44.420158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:44.420158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:44.420158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:44.420158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:44.420158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:44.420158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:44.420158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:44.420158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:44.420158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:44.420158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:13:44.420158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:13:44.420158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:13:44.420158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:13:44.422236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:13:44.422236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:13:44.422236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:13:44.422236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:13:44.422236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:44.422236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:44.422236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:44.422236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:44.422236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:44.422236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:44.422236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:44.422236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:44.422236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:44.422236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:44.422236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:44.422236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:44.423761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:44.423761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:44.423761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:44.423761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:44.423761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:13:44.423761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:13:44.423761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:13:44.423761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:13:44.423761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:13:44.423761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:13:44.423761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:13:44.423761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:13:44.423761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 625/1235] 10.222.0.88 () {56 vars in 1102 bytes} [Tue Aug 23 22:13:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:44.431891 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93617a2c-1997-4f58-941b-1823ca0fabd1 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:44.496495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:44.607330 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93617a2c-1997-4f58-941b-1823ca0fabd1 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:44.667588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56441a-a8b8-4546-b51f-18facd999377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:44.668238 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56441a-a8b8-4546-b51f-18facd999377 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:44.668517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56441a-a8b8-4546-b51f-18facd999377 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:44.668789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56441a-a8b8-4546-b51f-18facd999377 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:44.669534 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56441a-a8b8-4546-b51f-18facd999377 None admin] RBAC: Adding query filter params (name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:44.669839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56441a-a8b8-4546-b51f-18facd999377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:44.671444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56441a-a8b8-4546-b51f-18facd999377 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:44.683127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 611/1236] 10.222.0.88 () {56 vars in 1103 bytes} [Tue Aug 23 22:13:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:44.691271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb0500-b4bc-452b-9fa4-2ac056eebe2f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:44.831749 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:44.853594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb0500-b4bc-452b-9fa4-2ac056eebe2f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:44.923367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44de10-5f9c-47ad-8585-58885bf49881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:44.924526 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44de10-5f9c-47ad-8585-58885bf49881 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:44.925131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44de10-5f9c-47ad-8585-58885bf49881 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:44.925583 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44de10-5f9c-47ad-8585-58885bf49881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:44.946506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44de10-5f9c-47ad-8585-58885bf49881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:44.948197 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44de10-5f9c-47ad-8585-58885bf49881 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:44.979559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-2b44de10-5f9c-47ad-8585-58885bf49881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:44.979559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:44.979559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:44.979559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:44.979559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:44.979559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:44.979559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:44.979559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:44.979559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:44.979559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:44.979559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:44.979559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:44.979559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:13:44.979559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:13:44.979559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:13:44.979559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:44.988764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:13:44.989853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:13:44.989853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:44.989853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:13:44.989853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 626/1237] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:44.992720 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56441a-a8b8-4546-b51f-18facd999377 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:45.177843 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56441a-a8b8-4546-b51f-18facd999377 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:45.192477 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:45.235592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e26016-c891-4302-a2dd-4964a654cd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:45.236455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e26016-c891-4302-a2dd-4964a654cd5f None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:45.236836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e26016-c891-4302-a2dd-4964a654cd5f None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:45.237282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e26016-c891-4302-a2dd-4964a654cd5f None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:45.238084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e26016-c891-4302-a2dd-4964a654cd5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:45.238548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e26016-c891-4302-a2dd-4964a654cd5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:45.241292 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e26016-c891-4302-a2dd-4964a654cd5f None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:45.263801 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 612/1238] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:45.270794 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44de10-5f9c-47ad-8585-58885bf49881 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:45.414361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44de10-5f9c-47ad-8585-58885bf49881 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:45.469523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9173b07-38b8-4b83-b548-6fe919cae1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:45.470594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9173b07-38b8-4b83-b548-6fe919cae1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:45.471163 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9173b07-38b8-4b83-b548-6fe919cae1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:45.471735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9173b07-38b8-4b83-b548-6fe919cae1d8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:45.477198 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:45.488748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9173b07-38b8-4b83-b548-6fe919cae1d8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:45.492044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9173b07-38b8-4b83-b548-6fe919cae1d8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:45.515072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-b9173b07-38b8-4b83-b548-6fe919cae1d8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:13:45.515072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:45.515072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:45.515072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:45.515072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:45.515072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:45.515072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:45.515072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:45.515072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:45.515072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:45.515072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:45.515072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:45.515072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:13:45.515072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:13:45.515072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:13:45.515072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:45.517173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:45.518984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:45.518984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:45.518984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:13:45.518984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:13:45.518984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:13:45.518984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:13:45.518984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:13:45.518984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:13:45.518984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:13:45.518984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:13:45.518984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 627/1239] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:45.525101 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e26016-c891-4302-a2dd-4964a654cd5f None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:45.692268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e26016-c891-4302-a2dd-4964a654cd5f None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:45.747703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d39c8d3-08dc-4e64-8010-3dbfcd7353eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:45.748356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39c8d3-08dc-4e64-8010-3dbfcd7353eb None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:45.748718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39c8d3-08dc-4e64-8010-3dbfcd7353eb None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:45.749138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39c8d3-08dc-4e64-8010-3dbfcd7353eb None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:45.750429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d39c8d3-08dc-4e64-8010-3dbfcd7353eb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:45.750958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d39c8d3-08dc-4e64-8010-3dbfcd7353eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:45.754417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d39c8d3-08dc-4e64-8010-3dbfcd7353eb None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:45.780797 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 613/1240] 10.222.0.88 () {56 vars in 1143 bytes} [Tue Aug 23 22:13:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:45.789570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9173b07-38b8-4b83-b548-6fe919cae1d8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:45.944762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9173b07-38b8-4b83-b548-6fe919cae1d8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:45.998910 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfda65-918b-4d47-838d-b6d00cb2599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:45.999619 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfda65-918b-4d47-838d-b6d00cb2599d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:46.000116 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfda65-918b-4d47-838d-b6d00cb2599d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:46.000502 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfda65-918b-4d47-838d-b6d00cb2599d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:46.120734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:46.135267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfda65-918b-4d47-838d-b6d00cb2599d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:46.138184 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfda65-918b-4d47-838d-b6d00cb2599d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:46.344556 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 628/1241] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:13:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:13:46.354174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d39c8d3-08dc-4e64-8010-3dbfcd7353eb None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:46.408941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:46.549900 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d39c8d3-08dc-4e64-8010-3dbfcd7353eb None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:46.630632 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d07a9-4df7-42d9-856d-5e2f5caafc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:46.633181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d07a9-4df7-42d9-856d-5e2f5caafc17 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:46.633870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d07a9-4df7-42d9-856d-5e2f5caafc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:46.634690 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d07a9-4df7-42d9-856d-5e2f5caafc17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:46.666684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d07a9-4df7-42d9-856d-5e2f5caafc17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:46.671360 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d07a9-4df7-42d9-856d-5e2f5caafc17 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:46.700436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-6d1d07a9-4df7-42d9-856d-5e2f5caafc17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:46.700436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:46.700436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:46.700436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:46.700436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:46.700436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:46.700436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:46.700436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:46.700436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:46.700436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:46.700436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:46.700436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:46.700436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:13:46.700436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:13:46.700436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:13:46.700436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:46.702630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:13:46.704099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:13:46.704099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:46.704099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:13:46.704099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 614/1242] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:46.710046 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfda65-918b-4d47-838d-b6d00cb2599d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:46.885307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:46.897448 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfda65-918b-4d47-838d-b6d00cb2599d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:46.956617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0b5bf-ca9a-426d-9fc5-8eee6735e7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:46.957241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0b5bf-ca9a-426d-9fc5-8eee6735e7df None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:46.957520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0b5bf-ca9a-426d-9fc5-8eee6735e7df None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:46.957792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0b5bf-ca9a-426d-9fc5-8eee6735e7df None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:46.958455 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0b5bf-ca9a-426d-9fc5-8eee6735e7df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:46.958733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0b5bf-ca9a-426d-9fc5-8eee6735e7df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:46.960972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0b5bf-ca9a-426d-9fc5-8eee6735e7df None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:46.974224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 629/1243] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:13:46.981637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d07a9-4df7-42d9-856d-5e2f5caafc17 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:47.102664 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:47.135994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d07a9-4df7-42d9-856d-5e2f5caafc17 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:47.201898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0a12f-1b9c-4b92-aac5-b47497d8a428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:47.203028 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0a12f-1b9c-4b92-aac5-b47497d8a428 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:47.203599 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0a12f-1b9c-4b92-aac5-b47497d8a428 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:47.204080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0a12f-1b9c-4b92-aac5-b47497d8a428 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:47.218697 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0a12f-1b9c-4b92-aac5-b47497d8a428 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:47.221974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0a12f-1b9c-4b92-aac5-b47497d8a428 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:47.237667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-56a0a12f-1b9c-4b92-aac5-b47497d8a428 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:13:47.237667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:47.237667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:47.237667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:47.237667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:47.237667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:47.237667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:47.237667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:47.237667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:47.237667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:47.237667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:47.237667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:47.237667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:13:47.237667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:13:47.237667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:47.239728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:47.241718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:13:47.241718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:13:47.241718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:13:47.241718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:13:47.241718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:13:47.241718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:13:47.241718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:13:47.241718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:13:47.241718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 615/1244] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:13:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:47.246174 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0b5bf-ca9a-426d-9fc5-8eee6735e7df None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:47.389256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0b5bf-ca9a-426d-9fc5-8eee6735e7df None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:47.459917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b5769-cdbc-4681-aebe-79def5f0646d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:47.460887 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b5769-cdbc-4681-aebe-79def5f0646d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:47.461413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b5769-cdbc-4681-aebe-79def5f0646d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:47.461852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b5769-cdbc-4681-aebe-79def5f0646d None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:47.463197 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b5769-cdbc-4681-aebe-79def5f0646d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:47.463756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b5769-cdbc-4681-aebe-79def5f0646d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:47.465985 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b5769-cdbc-4681-aebe-79def5f0646d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:47.487037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 630/1245] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:13:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:13:47.497609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0a12f-1b9c-4b92-aac5-b47497d8a428 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:47.708245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0a12f-1b9c-4b92-aac5-b47497d8a428 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:47.759287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21cfd80-9b3f-41dd-b9d6-eeaa65bde953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:47.759921 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cfd80-9b3f-41dd-b9d6-eeaa65bde953 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:47.760401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cfd80-9b3f-41dd-b9d6-eeaa65bde953 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:47.760699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cfd80-9b3f-41dd-b9d6-eeaa65bde953 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:47.763018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21cfd80-9b3f-41dd-b9d6-eeaa65bde953 None admin] RBAC: Adding query filter params (role.id=23ea214daa734346a6da3a0a8c50d23a, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=e642dd7a07ca4afaa40c2769299d8bad) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:47.763018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21cfd80-9b3f-41dd-b9d6-eeaa65bde953 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:47.766195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21cfd80-9b3f-41dd-b9d6-eeaa65bde953 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:47.774124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 616/1246] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:13:47 2022] GET /identity/v3/role_assignments?user.id=e642dd7a07ca4afaa40c2769299d8bad&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=23ea214daa734346a6da3a0a8c50d23a => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:50.309154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66595c4-cd67-45d9-8446-b790779715e5 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:50.309751 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66595c4-cd67-45d9-8446-b790779715e5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:50.309751 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66595c4-cd67-45d9-8446-b790779715e5 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:50.311349 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 631/1247] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:13:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:13:50.317790 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeab042-8288-44cc-b4dd-2f09e7289fc3 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:50.319567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeab042-8288-44cc-b4dd-2f09e7289fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:50.319567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeab042-8288-44cc-b4dd-2f09e7289fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:50.327316 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:50.352914 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5aeab042-8288-44cc-b4dd-2f09e7289fc3 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:13:50.355138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5aeab042-8288-44cc-b4dd-2f09e7289fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:13:50.455421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 617/1248] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:13:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:13:50.812207 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66595c4-cd67-45d9-8446-b790779715e5 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:50.843861 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:51.009579 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66595c4-cd67-45d9-8446-b790779715e5 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:51.074328 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83fed6-5135-4ad0-a736-501b7a6b0f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:51.075091 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83fed6-5135-4ad0-a736-501b7a6b0f8f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:51.075456 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83fed6-5135-4ad0-a736-501b7a6b0f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:51.075931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83fed6-5135-4ad0-a736-501b7a6b0f8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:51.092714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83fed6-5135-4ad0-a736-501b7a6b0f8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:13:51.099313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-5a83fed6-5135-4ad0-a736-501b7a6b0f8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:13:51.099313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:51.099313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:51.099313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:51.099313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:51.099313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:51.099313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:51.099313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:51.099313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:51.099313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:51.099313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:51.099313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:51.099313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:13:51.099313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:13:51.099313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:13:51.099313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:13:51.101857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:13:51.101857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:13:51.101857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:13:51.101857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:13:51.101857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:51.101857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:51.101857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:51.101857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:51.101857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:51.101857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:51.101857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:51.101857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:51.101857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:51.101857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:51.101857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:51.101857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:51.103671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:51.103671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:51.103671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:51.103671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:51.103671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:13:51.103671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:13:51.103671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:13:51.103671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:13:51.103671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:13:51.103671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:13:51.103671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:13:51.103671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:13:51.103671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 632/1249] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:13:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:13:51.109568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeab042-8288-44cc-b4dd-2f09e7289fc3 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:51.281038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeab042-8288-44cc-b4dd-2f09e7289fc3 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:51.331089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:51.338751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85413ed1-5dd0-4114-b16d-f9321d2193d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:51.339586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85413ed1-5dd0-4114-b16d-f9321d2193d3 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:51.340064 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85413ed1-5dd0-4114-b16d-f9321d2193d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:51.340453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85413ed1-5dd0-4114-b16d-f9321d2193d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:51.341381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85413ed1-5dd0-4114-b16d-f9321d2193d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:51.341873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85413ed1-5dd0-4114-b16d-f9321d2193d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:51.343495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85413ed1-5dd0-4114-b16d-f9321d2193d3 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:51.358950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 618/1250] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:13:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:51.374726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83fed6-5135-4ad0-a736-501b7a6b0f8f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:51.533320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:51.577403 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83fed6-5135-4ad0-a736-501b7a6b0f8f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:51.649973 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5c74c-7976-498e-946d-1e92d0fb53ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:51.651003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5c74c-7976-498e-946d-1e92d0fb53ee None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:51.651475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5c74c-7976-498e-946d-1e92d0fb53ee None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:51.651952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5c74c-7976-498e-946d-1e92d0fb53ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:51.671359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5c74c-7976-498e-946d-1e92d0fb53ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:51.673206 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5c74c-7976-498e-946d-1e92d0fb53ee None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:51.700852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-30c5c74c-7976-498e-946d-1e92d0fb53ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:51.700852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:51.700852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:51.700852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:51.700852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:51.700852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:51.700852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:51.700852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:51.700852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:51.700852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:51.700852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:51.700852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:51.700852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:13:51.700852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:13:51.700852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:13:51.700852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:51.702776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:13:51.703797 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:13:51.703797 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:51.703797 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:13:51.703797 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 633/1251] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:51.711156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85413ed1-5dd0-4114-b16d-f9321d2193d3 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:51.866469 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85413ed1-5dd0-4114-b16d-f9321d2193d3 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:51.931335 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6767e72-42e0-462c-ab01-00a2ec9585ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:51.932557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6767e72-42e0-462c-ab01-00a2ec9585ed None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:51.933089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6767e72-42e0-462c-ab01-00a2ec9585ed None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:51.933575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6767e72-42e0-462c-ab01-00a2ec9585ed None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:51.934790 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6767e72-42e0-462c-ab01-00a2ec9585ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:51.935117 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6767e72-42e0-462c-ab01-00a2ec9585ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:51.937146 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6767e72-42e0-462c-ab01-00a2ec9585ed None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:51.959554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 619/1252] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:13:51.969526 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5c74c-7976-498e-946d-1e92d0fb53ee None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:52.157891 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5c74c-7976-498e-946d-1e92d0fb53ee None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:52.204267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:52.216166 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cf352-e4da-475b-9eb0-d019cc579f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:52.217209 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cf352-e4da-475b-9eb0-d019cc579f4e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:52.217721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cf352-e4da-475b-9eb0-d019cc579f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:52.218065 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cf352-e4da-475b-9eb0-d019cc579f4e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:52.234468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953cf352-e4da-475b-9eb0-d019cc579f4e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:52.235987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953cf352-e4da-475b-9eb0-d019cc579f4e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:52.257529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-953cf352-e4da-475b-9eb0-d019cc579f4e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:13:52.257529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:52.257529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:52.257529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:52.257529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:52.257529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:52.257529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:52.257529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:52.257529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:52.257529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:52.257529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:52.257529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:52.257529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:13:52.257529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:13:52.257529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:13:52.257529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:52.259558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:52.261108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:52.261108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:52.261108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:13:52.261108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:13:52.261108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:13:52.261108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:13:52.261108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:13:52.261108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:13:52.261108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:13:52.261108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:13:52.261108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 634/1253] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:52.267525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6767e72-42e0-462c-ab01-00a2ec9585ed None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:52.375136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:52.464361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6767e72-42e0-462c-ab01-00a2ec9585ed None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:52.521802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736c3e8-a91a-4977-bebf-cd001a8a5c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:52.522980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736c3e8-a91a-4977-bebf-cd001a8a5c9a None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:52.523461 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736c3e8-a91a-4977-bebf-cd001a8a5c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:52.523982 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736c3e8-a91a-4977-bebf-cd001a8a5c9a None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:52.525253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736c3e8-a91a-4977-bebf-cd001a8a5c9a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:52.525805 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736c3e8-a91a-4977-bebf-cd001a8a5c9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:52.530760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736c3e8-a91a-4977-bebf-cd001a8a5c9a None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:52.569104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 620/1254] 10.222.0.88 () {56 vars in 1143 bytes} [Tue Aug 23 22:13:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:13:52.577795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cf352-e4da-475b-9eb0-d019cc579f4e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:52.776501 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cf352-e4da-475b-9eb0-d019cc579f4e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:52.838527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbef1c-d21b-4fd5-ac2d-d2e172c48a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:52.839155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbef1c-d21b-4fd5-ac2d-d2e172c48a32 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:52.839446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbef1c-d21b-4fd5-ac2d-d2e172c48a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:52.839726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbef1c-d21b-4fd5-ac2d-d2e172c48a32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:52.946617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:52.984678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fbef1c-d21b-4fd5-ac2d-d2e172c48a32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:52.988835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fbef1c-d21b-4fd5-ac2d-d2e172c48a32 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:53.191876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 635/1255] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:13:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:13:53.200967 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736c3e8-a91a-4977-bebf-cd001a8a5c9a None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:53.382989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736c3e8-a91a-4977-bebf-cd001a8a5c9a None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:53.412958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:53.453588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428bc6d-e545-4419-adef-227cab29fb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:53.454504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428bc6d-e545-4419-adef-227cab29fb12 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:53.454967 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428bc6d-e545-4419-adef-227cab29fb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:53.455417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428bc6d-e545-4419-adef-227cab29fb12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:53.473951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d428bc6d-e545-4419-adef-227cab29fb12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:53.475535 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d428bc6d-e545-4419-adef-227cab29fb12 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:53.493715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-d428bc6d-e545-4419-adef-227cab29fb12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:53.493715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:53.493715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:53.493715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:53.493715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:53.493715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:53.493715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:53.493715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:53.493715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:53.493715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:53.493715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:53.493715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:53.493715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:13:53.493715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:13:53.493715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:13:53.493715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:53.496176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:13:53.498432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:13:53.498432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:53.498432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:13:53.498432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 621/1256] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:13:53.502397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbef1c-d21b-4fd5-ac2d-d2e172c48a32 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:53.671935 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:53.697526 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbef1c-d21b-4fd5-ac2d-d2e172c48a32 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:53.766106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e2367-6b8c-4669-be1f-5c6585cd9a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:53.767275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e2367-6b8c-4669-be1f-5c6585cd9a40 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:53.767863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e2367-6b8c-4669-be1f-5c6585cd9a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:53.768295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e2367-6b8c-4669-be1f-5c6585cd9a40 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:53.769163 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e2367-6b8c-4669-be1f-5c6585cd9a40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:53.769810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e2367-6b8c-4669-be1f-5c6585cd9a40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:53.772973 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e2367-6b8c-4669-be1f-5c6585cd9a40 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:53.790146 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 636/1257] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:53.797755 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428bc6d-e545-4419-adef-227cab29fb12 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:53.962171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428bc6d-e545-4419-adef-227cab29fb12 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:54.023847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7976a-b1db-4f4b-bfb8-60ae45eda124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:54.024472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7976a-b1db-4f4b-bfb8-60ae45eda124 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:54.024834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7976a-b1db-4f4b-bfb8-60ae45eda124 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:54.025238 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7976a-b1db-4f4b-bfb8-60ae45eda124 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:54.035743 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7976a-b1db-4f4b-bfb8-60ae45eda124 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:54.038857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7976a-b1db-4f4b-bfb8-60ae45eda124 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:54.056243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-dda7976a-b1db-4f4b-bfb8-60ae45eda124 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:13:54.056243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:54.056243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:54.056243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:54.056243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:54.056243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:54.056243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:54.056243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:54.056243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:54.056243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:54.056243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:54.056243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:54.056243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:13:54.056243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:13:54.056243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:54.058561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:54.060208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:13:54.060208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:13:54.060208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:13:54.060208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:13:54.060208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:13:54.060208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:13:54.060208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:13:54.060208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:13:54.060208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 622/1258] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:13:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:13:54.065339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e2367-6b8c-4669-be1f-5c6585cd9a40 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:54.244036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e2367-6b8c-4669-be1f-5c6585cd9a40 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:54.311364 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875289a4-a65e-4110-a912-f0d254f5597a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:54.312328 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875289a4-a65e-4110-a912-f0d254f5597a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:54.312822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875289a4-a65e-4110-a912-f0d254f5597a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:54.313343 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875289a4-a65e-4110-a912-f0d254f5597a None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:54.314629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875289a4-a65e-4110-a912-f0d254f5597a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:54.315166 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875289a4-a65e-4110-a912-f0d254f5597a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:54.317893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875289a4-a65e-4110-a912-f0d254f5597a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:54.342843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 637/1259] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:13:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:54.352988 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7976a-b1db-4f4b-bfb8-60ae45eda124 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:54.397287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:54.538410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7976a-b1db-4f4b-bfb8-60ae45eda124 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:54.592115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6e574-3123-4bb8-86fa-34e361f943b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:54.592606 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6e574-3123-4bb8-86fa-34e361f943b5 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:54.592890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6e574-3123-4bb8-86fa-34e361f943b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:54.593293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6e574-3123-4bb8-86fa-34e361f943b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:54.593872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf6e574-3123-4bb8-86fa-34e361f943b5 None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=e642dd7a07ca4afaa40c2769299d8bad) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:54.594176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf6e574-3123-4bb8-86fa-34e361f943b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:54.595697 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf6e574-3123-4bb8-86fa-34e361f943b5 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:54.613175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 623/1260] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:13:54 2022] GET /identity/v3/role_assignments?user.id=e642dd7a07ca4afaa40c2769299d8bad&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=dc5d71ab685e420697f520aa3a368177 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:13:57.054470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e690f-c500-44b7-82e1-041c4a6772dc None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:57.055052 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e690f-c500-44b7-82e1-041c4a6772dc None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:57.055052 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e690f-c500-44b7-82e1-041c4a6772dc None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:57.056602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 638/1261] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:13:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:13:57.062750 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7882a87-e0ff-4274-8173-650519fcb41f None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:57.063629 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7882a87-e0ff-4274-8173-650519fcb41f None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:57.063629 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7882a87-e0ff-4274-8173-650519fcb41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:57.086540 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-b7882a87-e0ff-4274-8173-650519fcb41f None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:13:57.087839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-b7882a87-e0ff-4274-8173-650519fcb41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:13:57.176430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 624/1262] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:13:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:13:57.538181 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e690f-c500-44b7-82e1-041c4a6772dc None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:57.558199 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:57.726349 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e690f-c500-44b7-82e1-041c4a6772dc None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:57.791740 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f314adc-ab56-4c66-9a4e-27e4fbb9acf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:57.792306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f314adc-ab56-4c66-9a4e-27e4fbb9acf9 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:57.792591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f314adc-ab56-4c66-9a4e-27e4fbb9acf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:57.792867 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f314adc-ab56-4c66-9a4e-27e4fbb9acf9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:57.805771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f314adc-ab56-4c66-9a4e-27e4fbb9acf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:13:57.811139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-6f314adc-ab56-4c66-9a4e-27e4fbb9acf9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:13:57.811139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:57.811139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:57.811139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:57.811139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:57.811139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:57.811139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:57.811139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:57.811139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:57.811139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:57.811139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:57.811139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:57.811139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:13:57.811139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:13:57.811139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:13:57.811139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:13:57.813287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:13:57.813287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:13:57.813287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:13:57.813287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:13:57.813287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:57.813287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:57.813287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:57.813287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:57.813287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:57.813287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:57.813287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:57.813287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:57.813287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:57.813287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:57.813287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:57.813287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:57.815082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:57.815082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:57.815082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:57.815082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:57.815082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:13:57.815082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:13:57.815082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:13:57.815082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:13:57.815082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:13:57.815082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:13:57.815082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:13:57.815082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:13:57.815082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 639/1263] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:13:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:57.820018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7882a87-e0ff-4274-8173-650519fcb41f None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:57.846027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:57.961666 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7882a87-e0ff-4274-8173-650519fcb41f None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:58.014570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3acb31-a1ea-4fa5-b8d7-4923bad90b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:58.015615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3acb31-a1ea-4fa5-b8d7-4923bad90b9d None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:58.016116 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3acb31-a1ea-4fa5-b8d7-4923bad90b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:58.016615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3acb31-a1ea-4fa5-b8d7-4923bad90b9d None admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:58.017825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3acb31-a1ea-4fa5-b8d7-4923bad90b9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:58.018135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3acb31-a1ea-4fa5-b8d7-4923bad90b9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:58.020601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3acb31-a1ea-4fa5-b8d7-4923bad90b9d None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:58.035991 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 625/1264] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:13:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:13:58.042544 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f314adc-ab56-4c66-9a4e-27e4fbb9acf9 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:58.132915 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:58.198013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f314adc-ab56-4c66-9a4e-27e4fbb9acf9 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:58.263381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c83d4-1872-4cec-b028-1d8583f80330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:58.264139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c83d4-1872-4cec-b028-1d8583f80330 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:58.264426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c83d4-1872-4cec-b028-1d8583f80330 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:58.264882 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c83d4-1872-4cec-b028-1d8583f80330 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:58.283418 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c83d4-1872-4cec-b028-1d8583f80330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:58.285258 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c83d4-1872-4cec-b028-1d8583f80330 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:58.308053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-569c83d4-1872-4cec-b028-1d8583f80330 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:58.308053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:58.308053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:58.308053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:58.308053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:58.308053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:58.308053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:58.308053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:58.308053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:58.308053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:58.308053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:58.308053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:58.308053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:13:58.308053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:13:58.308053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:13:58.308053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:58.309988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:13:58.312011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:13:58.312011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:58.312011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:13:58.312011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 640/1265] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:13:58.317134 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3acb31-a1ea-4fa5-b8d7-4923bad90b9d None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:58.469449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:58.505675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3acb31-a1ea-4fa5-b8d7-4923bad90b9d None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:58.549637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f00ff-2687-4bab-8a98-798aabb5e9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:58.550250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f00ff-2687-4bab-8a98-798aabb5e9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:58.550529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f00ff-2687-4bab-8a98-798aabb5e9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:58.550796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f00ff-2687-4bab-8a98-798aabb5e9a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:58.551456 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f00ff-2687-4bab-8a98-798aabb5e9a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:58.551753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f00ff-2687-4bab-8a98-798aabb5e9a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:58.553736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f00ff-2687-4bab-8a98-798aabb5e9a1 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:58.567715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 626/1266] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:58.578970 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c83d4-1872-4cec-b028-1d8583f80330 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:58.714834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c83d4-1872-4cec-b028-1d8583f80330 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:58.738361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:58.756848 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec14503-a94f-4d95-b803-9548c94b43a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:58.757913 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec14503-a94f-4d95-b803-9548c94b43a6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:58.758200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec14503-a94f-4d95-b803-9548c94b43a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:58.758472 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec14503-a94f-4d95-b803-9548c94b43a6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:58.772679 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec14503-a94f-4d95-b803-9548c94b43a6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:58.774190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec14503-a94f-4d95-b803-9548c94b43a6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:58.786372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-6ec14503-a94f-4d95-b803-9548c94b43a6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:13:58.786372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:58.786372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:58.786372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:58.786372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:58.786372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:58.786372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:58.786372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:58.786372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:58.786372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:58.786372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:58.786372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:58.786372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:13:58.786372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:13:58.786372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:13:58.786372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:58.788763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:58.790728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:58.790728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:58.790728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:13:58.790728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:13:58.790728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:13:58.790728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:13:58.790728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:13:58.790728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:13:58.790728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:13:58.790728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:13:58.790728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 641/1267] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:58.794375 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f00ff-2687-4bab-8a98-798aabb5e9a1 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:58.959656 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f00ff-2687-4bab-8a98-798aabb5e9a1 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:59.031831 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650a740-3473-40ad-81af-d2ab3b379369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:59.032773 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650a740-3473-40ad-81af-d2ab3b379369 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:59.033204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650a740-3473-40ad-81af-d2ab3b379369 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:59.033669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650a740-3473-40ad-81af-d2ab3b379369 None admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:59.034881 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650a740-3473-40ad-81af-d2ab3b379369 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:59.035406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650a740-3473-40ad-81af-d2ab3b379369 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:59.038161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650a740-3473-40ad-81af-d2ab3b379369 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:59.049436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:59.064844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 627/1268] 10.222.0.88 () {56 vars in 1143 bytes} [Tue Aug 23 22:13:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:59.077711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec14503-a94f-4d95-b803-9548c94b43a6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:59.205615 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec14503-a94f-4d95-b803-9548c94b43a6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:59.245182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f764c1f-bfbb-4805-b7fa-da0c28c39b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:59.245702 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f764c1f-bfbb-4805-b7fa-da0c28c39b87 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:59.246009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f764c1f-bfbb-4805-b7fa-da0c28c39b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:59.246341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f764c1f-bfbb-4805-b7fa-da0c28c39b87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:59.258664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f764c1f-bfbb-4805-b7fa-da0c28c39b87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:59.260196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f764c1f-bfbb-4805-b7fa-da0c28c39b87 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:59.277436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-8f764c1f-bfbb-4805-b7fa-da0c28c39b87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:59.277436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:59.277436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:59.277436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:59.277436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:59.277436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:59.277436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:59.277436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:59.277436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:59.277436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:59.277436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:59.277436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:59.277436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:13:59.277436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:13:59.277436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:13:59.277436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:59.279492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:13:59.281781 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:13:59.281781 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:59.281781 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:13:59.281781 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 642/1269] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:59.289811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650a740-3473-40ad-81af-d2ab3b379369 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:59.492398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650a740-3473-40ad-81af-d2ab3b379369 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:59.546097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85ffa9-e9e6-4b6a-b199-343162b7228c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:59.546734 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85ffa9-e9e6-4b6a-b199-343162b7228c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:59.547013 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85ffa9-e9e6-4b6a-b199-343162b7228c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:59.547451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85ffa9-e9e6-4b6a-b199-343162b7228c None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:59.548418 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85ffa9-e9e6-4b6a-b199-343162b7228c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:59.548774 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85ffa9-e9e6-4b6a-b199-343162b7228c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:59.550272 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85ffa9-e9e6-4b6a-b199-343162b7228c None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:59.566905 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 628/1270] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:59.573803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f764c1f-bfbb-4805-b7fa-da0c28c39b87 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:59.627000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:59.719719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f764c1f-bfbb-4805-b7fa-da0c28c39b87 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:59.765164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0165a6-3dac-4abc-8ca0-c99e08e3d9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:59.766147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0165a6-3dac-4abc-8ca0-c99e08e3d9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:59.766625 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0165a6-3dac-4abc-8ca0-c99e08e3d9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:59.767090 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0165a6-3dac-4abc-8ca0-c99e08e3d9e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:59.777701 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0165a6-3dac-4abc-8ca0-c99e08e3d9e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:59.779319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0165a6-3dac-4abc-8ca0-c99e08e3d9e7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:59.800103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-fa0165a6-3dac-4abc-8ca0-c99e08e3d9e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:13:59.800103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:59.800103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:59.800103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:59.800103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:59.800103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:59.800103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:59.800103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:59.800103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:59.800103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:59.800103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:59.800103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:59.800103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:13:59.800103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:13:59.800103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:59.802233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:59.805079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:13:59.805079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:13:59.805079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:13:59.805079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:13:59.805079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:13:59.805079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:13:59.805079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:13:59.805079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:13:59.805079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 643/1271] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:13:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:13:59.808804 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85ffa9-e9e6-4b6a-b199-343162b7228c None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:59.910927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:59.982879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85ffa9-e9e6-4b6a-b199-343162b7228c None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:00.046876 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a654b2-51d8-4982-addc-5518c96cc7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:00.047592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a654b2-51d8-4982-addc-5518c96cc7f0 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:00.047876 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a654b2-51d8-4982-addc-5518c96cc7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:00.048150 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a654b2-51d8-4982-addc-5518c96cc7f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:00.049003 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a654b2-51d8-4982-addc-5518c96cc7f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:00.049416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a654b2-51d8-4982-addc-5518c96cc7f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:00.050912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a654b2-51d8-4982-addc-5518c96cc7f0 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:00.073504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 629/1272] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:13:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:00.082868 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0165a6-3dac-4abc-8ca0-c99e08e3d9e7 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:00.232149 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:00.245340 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0165a6-3dac-4abc-8ca0-c99e08e3d9e7 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:00.295877 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e3827-44d3-4417-b1e4-6cb37f510533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:00.296717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e3827-44d3-4417-b1e4-6cb37f510533 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:00.297129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e3827-44d3-4417-b1e4-6cb37f510533 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:00.297515 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e3827-44d3-4417-b1e4-6cb37f510533 None admin] PATH_INFO: `/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/e642dd7a07ca4afaa40c2769299d8bad/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:00.299963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e3827-44d3-4417-b1e4-6cb37f510533 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a7317af325f485e8fae49631cb5bc20, user_id=e642dd7a07ca4afaa40c2769299d8bad, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:00.301609 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e3827-44d3-4417-b1e4-6cb37f510533 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:00.324735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 644/1273] 10.222.0.88 () {58 vars in 1439 bytes} [Tue Aug 23 22:14:00 2022] PUT /identity/v3/projects/8a7317af325f485e8fae49631cb5bc20/users/e642dd7a07ca4afaa40c2769299d8bad/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:14:02.724656 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317bb54-4838-4e61-87f3-847e0d7439a1 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:02.725281 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317bb54-4838-4e61-87f3-847e0d7439a1 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:02.725281 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317bb54-4838-4e61-87f3-847e0d7439a1 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:02.726600 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 630/1274] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:14:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:14:02.733745 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbba2a-259d-4763-81eb-19a64ed234c5 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:02.734530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbba2a-259d-4763-81eb-19a64ed234c5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:02.734727 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbba2a-259d-4763-81eb-19a64ed234c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:02.765438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-12dbba2a-259d-4763-81eb-19a64ed234c5 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:14:02.767557 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-12dbba2a-259d-4763-81eb-19a64ed234c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:14:02.862830 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 645/1275] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:14:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:14:03.184498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f317bb54-4838-4e61-87f3-847e0d7439a1 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:03.305193 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f317bb54-4838-4e61-87f3-847e0d7439a1 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:03.326015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:03.348633 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc4196-2083-42f8-ac45-6d32649ab2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:03.350278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc4196-2083-42f8-ac45-6d32649ab2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:03.350717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc4196-2083-42f8-ac45-6d32649ab2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:03.350916 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc4196-2083-42f8-ac45-6d32649ab2f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:03.364452 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc4196-2083-42f8-ac45-6d32649ab2f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:14:03.370023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-edfc4196-2083-42f8-ac45-6d32649ab2f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:14:03.370023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:03.370023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:03.370023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:03.370023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:03.370023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:03.370023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:03.370023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:03.370023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:03.370023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:03.370023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:03.370023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:03.370023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:14:03.370023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:14:03.370023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:14:03.370023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:14:03.371962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:14:03.371962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:14:03.371962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:14:03.371962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:14:03.371962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:03.371962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:03.371962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:03.371962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:03.371962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:03.371962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:03.371962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:03.371962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:03.371962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:03.371962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:03.371962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:03.371962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:03.373826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:03.373826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:03.373826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:03.373826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:03.373826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:14:03.373826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:14:03.373826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:14:03.373826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:14:03.373826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:14:03.373826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:14:03.373826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:14:03.373826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:14:03.373826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 631/1276] 10.222.0.88 () {56 vars in 1096 bytes} [Tue Aug 23 22:14:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:03.379582 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbba2a-259d-4763-81eb-19a64ed234c5 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:03.531131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:03.561697 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbba2a-259d-4763-81eb-19a64ed234c5 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:03.614240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe86e1-a1fa-4545-b921-c91f18310ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:03.615518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe86e1-a1fa-4545-b921-c91f18310ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:03.615810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe86e1-a1fa-4545-b921-c91f18310ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:03.616083 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe86e1-a1fa-4545-b921-c91f18310ab6 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:03.616792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe86e1-a1fa-4545-b921-c91f18310ab6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:03.617120 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe86e1-a1fa-4545-b921-c91f18310ab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:03.618551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe86e1-a1fa-4545-b921-c91f18310ab6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:03.628715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 646/1277] 10.222.0.88 () {56 vars in 1099 bytes} [Tue Aug 23 22:14:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:03.639779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc4196-2083-42f8-ac45-6d32649ab2f0 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:03.822963 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc4196-2083-42f8-ac45-6d32649ab2f0 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:03.882842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47112cfb-3fed-471c-b96f-952025771efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:03.883992 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47112cfb-3fed-471c-b96f-952025771efb None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:03.884376 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47112cfb-3fed-471c-b96f-952025771efb None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:03.884830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47112cfb-3fed-471c-b96f-952025771efb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:03.903701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47112cfb-3fed-471c-b96f-952025771efb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:03.906346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47112cfb-3fed-471c-b96f-952025771efb None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:03.933554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-47112cfb-3fed-471c-b96f-952025771efb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:03.933554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:03.933554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:03.933554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:03.933554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:03.933554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:03.933554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:03.933554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:03.933554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:03.933554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:03.933554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:03.933554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:03.933554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:14:03.933554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:14:03.933554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:14:03.933554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:03.934627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:14:03.935527 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:14:03.935527 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:03.935527 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:14:03.935527 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 632/1278] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:14:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:03.941976 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe86e1-a1fa-4545-b921-c91f18310ab6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:04.053869 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe86e1-a1fa-4545-b921-c91f18310ab6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:04.092892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc6eba-5c14-4dad-a445-7a0ed707b371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:04.093595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc6eba-5c14-4dad-a445-7a0ed707b371 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:04.093923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc6eba-5c14-4dad-a445-7a0ed707b371 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:04.094239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc6eba-5c14-4dad-a445-7a0ed707b371 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:04.094973 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc6eba-5c14-4dad-a445-7a0ed707b371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:04.095322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc6eba-5c14-4dad-a445-7a0ed707b371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:04.096755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc6eba-5c14-4dad-a445-7a0ed707b371 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:04.110944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 647/1279] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:04.118986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47112cfb-3fed-471c-b96f-952025771efb None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:04.176057 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:04.262455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47112cfb-3fed-471c-b96f-952025771efb None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:04.301736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3a962-d2cf-4157-a90e-249be4863491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:04.302653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3a962-d2cf-4157-a90e-249be4863491 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:04.302797 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3a962-d2cf-4157-a90e-249be4863491 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:04.303074 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3a962-d2cf-4157-a90e-249be4863491 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:04.312359 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3a962-d2cf-4157-a90e-249be4863491 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:04.313771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3a962-d2cf-4157-a90e-249be4863491 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:04.327012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-40c3a962-d2cf-4157-a90e-249be4863491 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:14:04.327012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:04.327012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:04.327012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:04.327012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:04.327012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:04.327012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:04.327012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:04.327012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:04.327012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:04.327012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:04.327012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:04.327012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:14:04.327012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:14:04.327012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:14:04.327012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:04.328941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:04.331067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:04.331067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:04.331067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:14:04.331067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:14:04.331067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:14:04.331067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:14:04.331067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:14:04.331067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:14:04.331067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:14:04.331067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:14:04.334027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 633/1280] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:14:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:14:04.335284 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc6eba-5c14-4dad-a445-7a0ed707b371 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:04.339692 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:04.481947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc6eba-5c14-4dad-a445-7a0ed707b371 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:04.543201 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177afc7-b79c-4277-9d56-1c2e988ec2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:04.544197 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177afc7-b79c-4277-9d56-1c2e988ec2ad None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:04.544679 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177afc7-b79c-4277-9d56-1c2e988ec2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:04.545167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177afc7-b79c-4277-9d56-1c2e988ec2ad None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:04.546534 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177afc7-b79c-4277-9d56-1c2e988ec2ad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:04.547086 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177afc7-b79c-4277-9d56-1c2e988ec2ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:04.549832 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177afc7-b79c-4277-9d56-1c2e988ec2ad None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:04.585460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 648/1281] 10.222.0.88 () {56 vars in 1143 bytes} [Tue Aug 23 22:14:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:04.592556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3a962-d2cf-4157-a90e-249be4863491 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:04.702495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:04.714802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3a962-d2cf-4157-a90e-249be4863491 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:04.764264 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131b41b-1d86-4b9e-8a97-b8d832a306cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:04.765353 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131b41b-1d86-4b9e-8a97-b8d832a306cd None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:04.765847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131b41b-1d86-4b9e-8a97-b8d832a306cd None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:04.766344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131b41b-1d86-4b9e-8a97-b8d832a306cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:04.855530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131b41b-1d86-4b9e-8a97-b8d832a306cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:04.857521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131b41b-1d86-4b9e-8a97-b8d832a306cd None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:04.963625 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:04.977702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 634/1282] 10.222.0.88 () {58 vars in 1282 bytes} [Tue Aug 23 22:14:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 22:14:04.984552 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177afc7-b79c-4277-9d56-1c2e988ec2ad None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:05.055751 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:05.125429 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177afc7-b79c-4277-9d56-1c2e988ec2ad None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:05.168005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dab4c1-7b5d-4aee-8d4f-ca7c05e119ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:05.168735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dab4c1-7b5d-4aee-8d4f-ca7c05e119ac None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:05.169108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dab4c1-7b5d-4aee-8d4f-ca7c05e119ac None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:05.169409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dab4c1-7b5d-4aee-8d4f-ca7c05e119ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:05.181731 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dab4c1-7b5d-4aee-8d4f-ca7c05e119ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:05.183279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dab4c1-7b5d-4aee-8d4f-ca7c05e119ac None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:05.199558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-79dab4c1-7b5d-4aee-8d4f-ca7c05e119ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:05.199558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:05.199558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:05.199558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:05.199558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:05.199558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:05.199558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:05.199558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:05.199558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:05.199558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:05.199558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:05.199558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:05.199558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:14:05.199558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:14:05.199558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:14:05.199558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:05.201817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:14:05.203965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:14:05.203965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:05.203965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:14:05.203965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 649/1283] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:14:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:05.207234 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131b41b-1d86-4b9e-8a97-b8d832a306cd None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:05.325789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131b41b-1d86-4b9e-8a97-b8d832a306cd None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:05.366170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae683e9-29dd-48c3-b263-b39861358290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:05.366851 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae683e9-29dd-48c3-b263-b39861358290 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:05.367168 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae683e9-29dd-48c3-b263-b39861358290 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:05.367480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae683e9-29dd-48c3-b263-b39861358290 None admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:05.368226 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae683e9-29dd-48c3-b263-b39861358290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:05.368569 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae683e9-29dd-48c3-b263-b39861358290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:05.370045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae683e9-29dd-48c3-b263-b39861358290 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:05.385802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 635/1284] 10.222.0.88 () {56 vars in 1109 bytes} [Tue Aug 23 22:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:05.391877 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dab4c1-7b5d-4aee-8d4f-ca7c05e119ac None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:05.540155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dab4c1-7b5d-4aee-8d4f-ca7c05e119ac None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:05.565660 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:05.603747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a24541-16be-4215-a9d2-8ec6008c42f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:05.604833 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a24541-16be-4215-a9d2-8ec6008c42f6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:05.605362 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a24541-16be-4215-a9d2-8ec6008c42f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:05.605879 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a24541-16be-4215-a9d2-8ec6008c42f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:05.619177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a24541-16be-4215-a9d2-8ec6008c42f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:05.621950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a24541-16be-4215-a9d2-8ec6008c42f6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:05.645222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-34a24541-16be-4215-a9d2-8ec6008c42f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:14:05.645222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:05.645222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:05.645222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:05.645222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:05.645222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:05.645222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:05.645222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:05.645222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:05.645222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:05.645222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:05.645222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:05.645222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:14:05.645222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:14:05.645222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:05.647178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:05.649339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:14:05.649339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:14:05.649339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:14:05.649339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:14:05.649339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:14:05.649339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:14:05.649339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:14:05.649339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 23 22:14:05.649339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 650/1285] 10.222.0.88 () {56 vars in 1111 bytes} [Tue Aug 23 22:14:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:14:05.658004 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae683e9-29dd-48c3-b263-b39861358290 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:05.727323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:05.821174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae683e9-29dd-48c3-b263-b39861358290 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:05.863719 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b3c6d-335b-4cd0-92aa-6bbd96b7332c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:05.864409 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b3c6d-335b-4cd0-92aa-6bbd96b7332c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:05.864632 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b3c6d-335b-4cd0-92aa-6bbd96b7332c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:05.865088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b3c6d-335b-4cd0-92aa-6bbd96b7332c None admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:05.866064 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b3c6d-335b-4cd0-92aa-6bbd96b7332c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:05.866376 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b3c6d-335b-4cd0-92aa-6bbd96b7332c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:05.867846 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b3c6d-335b-4cd0-92aa-6bbd96b7332c None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:05.881534 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 636/1286] 10.222.0.88 () {56 vars in 1148 bytes} [Tue Aug 23 22:14:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:05.889597 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a24541-16be-4215-a9d2-8ec6008c42f6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:06.027765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a24541-16be-4215-a9d2-8ec6008c42f6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:06.072355 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba7cbb0-2b18-42b1-b85e-8f33c7396bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:06.072981 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7cbb0-2b18-42b1-b85e-8f33c7396bed None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:06.073293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7cbb0-2b18-42b1-b85e-8f33c7396bed None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:06.073569 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7cbb0-2b18-42b1-b85e-8f33c7396bed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:06.074327 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7cbb0-2b18-42b1-b85e-8f33c7396bed None admin] RBAC: Adding query filter params (role.id=dc5d71ab685e420697f520aa3a368177, scope.project.id=8a7317af325f485e8fae49631cb5bc20, user.id=e642dd7a07ca4afaa40c2769299d8bad) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:06.074643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7cbb0-2b18-42b1-b85e-8f33c7396bed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:06.076104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7cbb0-2b18-42b1-b85e-8f33c7396bed None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:06.084081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 651/1287] 10.222.0.88 () {56 vars in 1374 bytes} [Tue Aug 23 22:14:05 2022] GET /identity/v3/role_assignments?user.id=e642dd7a07ca4afaa40c2769299d8bad&scope.project.id=8a7317af325f485e8fae49631cb5bc20&role.id=dc5d71ab685e420697f520aa3a368177 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:08.214606 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c087d1-319e-4133-9f3e-3705d03e1812 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:08.215213 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c087d1-319e-4133-9f3e-3705d03e1812 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:08.215213 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c087d1-319e-4133-9f3e-3705d03e1812 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:08.216567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 637/1288] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:14:08.222653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8639f-0faf-424a-a38a-a68f0908acdc None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:08.223230 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8639f-0faf-424a-a38a-a68f0908acdc None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:08.223230 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8639f-0faf-424a-a38a-a68f0908acdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:08.244788 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-76a8639f-0faf-424a-a38a-a68f0908acdc None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:14:08.246679 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-76a8639f-0faf-424a-a38a-a68f0908acdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:14:08.288889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:08.333359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 652/1289] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:14:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:14:08.622296 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c087d1-319e-4133-9f3e-3705d03e1812 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:08.794885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c087d1-319e-4133-9f3e-3705d03e1812 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:08.815959 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:08.844276 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa78ab-5b68-4e88-b5e4-7fdb59bcb681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:08.844922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa78ab-5b68-4e88-b5e4-7fdb59bcb681 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:08.845256 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa78ab-5b68-4e88-b5e4-7fdb59bcb681 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:08.845495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa78ab-5b68-4e88-b5e4-7fdb59bcb681 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:08.846083 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa78ab-5b68-4e88-b5e4-7fdb59bcb681 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:08.847461 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa78ab-5b68-4e88-b5e4-7fdb59bcb681 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-06fa78ab-5b68-4e88-b5e4-7fdb59bcb681 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:08.858564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:08.860813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:08.860813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:08.860813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:08.860813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:08.860813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:08.860813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:08.860813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:14:08.860813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:14:08.860813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:14:08.860813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:14:08.860813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:08.860813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:08.860813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:08.860813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:08.860813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:08.860813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:08.862675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:08.862675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:08.862675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:08.862675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:08.862675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:08.862675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:08.862675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:08.862675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:08.862675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:08.862675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:08.862675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:14:08.862675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:14:08.862675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:08.862675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:14:08.862675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 638/1290] 10.222.0.88 () {56 vars in 1117 bytes} [Tue Aug 23 22:14:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:08.868458 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8639f-0faf-424a-a38a-a68f0908acdc None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:09.020085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8639f-0faf-424a-a38a-a68f0908acdc None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:09.060944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70ffa8-329a-4ece-b8d8-449ed5e2553c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:09.061588 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70ffa8-329a-4ece-b8d8-449ed5e2553c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:09.061874 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70ffa8-329a-4ece-b8d8-449ed5e2553c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:09.062152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70ffa8-329a-4ece-b8d8-449ed5e2553c None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:09.062790 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70ffa8-329a-4ece-b8d8-449ed5e2553c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:09.063092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70ffa8-329a-4ece-b8d8-449ed5e2553c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:09.064477 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70ffa8-329a-4ece-b8d8-449ed5e2553c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:09.070369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 653/1291] 10.222.0.88 () {56 vars in 1116 bytes} [Tue Aug 23 22:14:08 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:14:09.077092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa78ab-5b68-4e88-b5e4-7fdb59bcb681 None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:09.197967 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa78ab-5b68-4e88-b5e4-7fdb59bcb681 None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:09.238460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ad003-805b-4c98-b824-aa952d648685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:09.239431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ad003-805b-4c98-b824-aa952d648685 None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:09.240067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ad003-805b-4c98-b824-aa952d648685 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:09.240249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ad003-805b-4c98-b824-aa952d648685 None admin] PATH_INFO: `/v3/services` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:09.240916 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ad003-805b-4c98-b824-aa952d648685 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:09.241250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ad003-805b-4c98-b824-aa952d648685 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:09.242636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ad003-805b-4c98-b824-aa952d648685 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:09.248102 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 639/1292] 10.222.0.88 () {56 vars in 1116 bytes} [Tue Aug 23 22:14:09 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:11.914315 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d5fdc-676d-4938-92cc-c66936263fad None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:11.914924 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d5fdc-676d-4938-92cc-c66936263fad None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:11.915367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d5fdc-676d-4938-92cc-c66936263fad None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:11.917698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 654/1293] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:14:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:14:11.927661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a2e3c-8b17-4e3c-a160-3a80d174e7f3 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:11.928647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a2e3c-8b17-4e3c-a160-3a80d174e7f3 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:11.928647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a2e3c-8b17-4e3c-a160-3a80d174e7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:11.958404 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-197a2e3c-8b17-4e3c-a160-3a80d174e7f3 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:14:11.960525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-197a2e3c-8b17-4e3c-a160-3a80d174e7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:14:11.998252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:12.046532 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 640/1294] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:14:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:14:12.327332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d5fdc-676d-4938-92cc-c66936263fad None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:12.386300 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:12.504160 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d5fdc-676d-4938-92cc-c66936263fad None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:12.558773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4608258-52f2-46da-8f5a-5f7347e5e41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:12.559379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4608258-52f2-46da-8f5a-5f7347e5e41f None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:12.559655 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4608258-52f2-46da-8f5a-5f7347e5e41f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:12.559924 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4608258-52f2-46da-8f5a-5f7347e5e41f None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:12.560575 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4608258-52f2-46da-8f5a-5f7347e5e41f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:12.561984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4608258-52f2-46da-8f5a-5f7347e5e41f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:12.580203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 655/1295] 10.222.0.88 () {60 vars in 1141 bytes} [Tue Aug 23 22:14:12 2022] POST /identity/v3/services => generated 246 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:14:14.737558 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dca766-6850-4023-8b63-0dc51324c5bf None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:14.738237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dca766-6850-4023-8b63-0dc51324c5bf None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:14.738537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dca766-6850-4023-8b63-0dc51324c5bf None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:14.741032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 641/1296] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:14:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:14:14.749524 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe42703-23c5-4c37-af04-89e03e8387e4 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:14.750248 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe42703-23c5-4c37-af04-89e03e8387e4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:14.750522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe42703-23c5-4c37-af04-89e03e8387e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:14.783235 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ffe42703-23c5-4c37-af04-89e03e8387e4 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:14:14.785430 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ffe42703-23c5-4c37-af04-89e03e8387e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:14:14.927817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 656/1297] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:14:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:14:15.228426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dca766-6850-4023-8b63-0dc51324c5bf None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:15.372724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dca766-6850-4023-8b63-0dc51324c5bf None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:15.424882 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f387f0-49e2-4352-9243-749532005c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:15.425677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f387f0-49e2-4352-9243-749532005c5d None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:15.425999 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f387f0-49e2-4352-9243-749532005c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:15.426356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f387f0-49e2-4352-9243-749532005c5d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:15.427842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f387f0-49e2-4352-9243-749532005c5d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:15.429568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f387f0-49e2-4352-9243-749532005c5d None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-99f387f0-49e2-4352-9243-749532005c5d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:15.441654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:15.444361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:15.444361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:15.444361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:15.444361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:15.444361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:15.444361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:15.444361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:14:15.444361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:14:15.444361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:14:15.444361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:14:15.444361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:15.444361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:15.444361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:15.444361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:15.444361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:15.444361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:15.446503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:15.446503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:15.446503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:15.446503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:15.446503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:15.446503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:15.446503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:15.446503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:15.446503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:15.446503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:15.446503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:14:15.446503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:14:15.446503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:15.446503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:14:15.446503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 642/1298] 10.222.0.88 () {56 vars in 1117 bytes} [Tue Aug 23 22:14:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:15.453107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe42703-23c5-4c37-af04-89e03e8387e4 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:15.478630 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:15.596438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe42703-23c5-4c37-af04-89e03e8387e4 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:15.639388 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02cde5-f358-4bd6-8646-f4489e1fd25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:15.640082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02cde5-f358-4bd6-8646-f4489e1fd25d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:15.640416 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02cde5-f358-4bd6-8646-f4489e1fd25d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:15.640726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02cde5-f358-4bd6-8646-f4489e1fd25d None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:15.641491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02cde5-f358-4bd6-8646-f4489e1fd25d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:15.641894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02cde5-f358-4bd6-8646-f4489e1fd25d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:15.643378 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02cde5-f358-4bd6-8646-f4489e1fd25d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:15.650383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 657/1299] 10.222.0.88 () {56 vars in 1116 bytes} [Tue Aug 23 22:14:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:15.660182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f387f0-49e2-4352-9243-749532005c5d None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:15.706285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:15.794108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f387f0-49e2-4352-9243-749532005c5d None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:15.835424 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0dfdd5-43f0-471b-a022-978c09934a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:15.836081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dfdd5-43f0-471b-a022-978c09934a4c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:15.836437 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dfdd5-43f0-471b-a022-978c09934a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:15.836749 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dfdd5-43f0-471b-a022-978c09934a4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:15.837519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0dfdd5-43f0-471b-a022-978c09934a4c None admin] RBAC: Adding query filter params (interface=public, service_id=19627e23ff344a9eba94dbb71277a346, region_id=RegionOne) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:15.837823 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0dfdd5-43f0-471b-a022-978c09934a4c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:15.839397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0dfdd5-43f0-471b-a022-978c09934a4c None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:15.845408 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 643/1300] 10.222.0.88 () {56 vars in 1251 bytes} [Tue Aug 23 22:14:15 2022] GET /identity/v3/endpoints?service_id=19627e23ff344a9eba94dbb71277a346&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:15.855212 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02cde5-f358-4bd6-8646-f4489e1fd25d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:15.968752 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:15.990770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02cde5-f358-4bd6-8646-f4489e1fd25d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:16.033504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95be637-e7a7-44ae-956f-07c56fc35258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:16.034285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95be637-e7a7-44ae-956f-07c56fc35258 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:16.034652 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95be637-e7a7-44ae-956f-07c56fc35258 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:16.035082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95be637-e7a7-44ae-956f-07c56fc35258 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:16.035802 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95be637-e7a7-44ae-956f-07c56fc35258 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:16.037312 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95be637-e7a7-44ae-956f-07c56fc35258 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:16.045788 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 658/1301] 10.222.0.88 () {56 vars in 1087 bytes} [Tue Aug 23 22:14:15 2022] GET /identity/v3/services => generated 2191 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:14:18.856002 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987624d-deea-4120-b093-d66b00b52c55 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:18.856558 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987624d-deea-4120-b093-d66b00b52c55 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:18.856991 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987624d-deea-4120-b093-d66b00b52c55 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:18.859573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 644/1302] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:14:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:14:18.867885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdce11-cabc-4e3e-ad7e-258df6714d10 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:18.868483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdce11-cabc-4e3e-ad7e-258df6714d10 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:18.868483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdce11-cabc-4e3e-ad7e-258df6714d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:18.895782 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e1cdce11-cabc-4e3e-ad7e-258df6714d10 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:14:18.897826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e1cdce11-cabc-4e3e-ad7e-258df6714d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:14:18.996976 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 659/1303] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:14:18 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:14:19.345293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987624d-deea-4120-b093-d66b00b52c55 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:19.488458 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:19.504326 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987624d-deea-4120-b093-d66b00b52c55 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:19.547624 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec636dbb-603d-4b59-800a-55cc1444263c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:19.548364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec636dbb-603d-4b59-800a-55cc1444263c None admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:19.548654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec636dbb-603d-4b59-800a-55cc1444263c None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:19.548977 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec636dbb-603d-4b59-800a-55cc1444263c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:19.549655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec636dbb-603d-4b59-800a-55cc1444263c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:19.551090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec636dbb-603d-4b59-800a-55cc1444263c None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application [None req-ec636dbb-603d-4b59-800a-55cc1444263c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:19.561594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:19.565602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:19.565602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:19.565602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:19.565602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:19.565602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:19.565602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:19.565602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:14:19.565602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:14:19.565602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:14:19.565602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:14:19.565602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:19.565602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:19.565602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:19.565602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:19.565602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:19.565602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:19.566739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:19.566739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:19.566739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:19.566739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:19.566739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:19.566739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:19.566739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:19.566739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:19.566739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:19.566739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:19.566739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:14:19.566739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:14:19.566739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:19.566739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: ERROR keystone.server.flask.application Aug 23 22:14:19.566739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 645/1304] 10.222.0.88 () {56 vars in 1117 bytes} [Tue Aug 23 22:14:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:19.570955 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdce11-cabc-4e3e-ad7e-258df6714d10 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:19.663264 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:19.690130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdce11-cabc-4e3e-ad7e-258df6714d10 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:19.729936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5374a-f818-43b2-bbd6-674aba756da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:19.730603 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5374a-f818-43b2-bbd6-674aba756da1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:19.730928 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5374a-f818-43b2-bbd6-674aba756da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:19.731247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5374a-f818-43b2-bbd6-674aba756da1 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:19.731950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5374a-f818-43b2-bbd6-674aba756da1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:19.732298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5374a-f818-43b2-bbd6-674aba756da1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:19.733811 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5374a-f818-43b2-bbd6-674aba756da1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:19.740827 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 660/1305] 10.222.0.88 () {56 vars in 1116 bytes} [Tue Aug 23 22:14:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:19.750032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec636dbb-603d-4b59-800a-55cc1444263c None admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:19.941709 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec636dbb-603d-4b59-800a-55cc1444263c None admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:19.986861 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2839060-3d8c-439b-b5b2-0bce15370005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:19.987537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2839060-3d8c-439b-b5b2-0bce15370005 None admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:19.987869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2839060-3d8c-439b-b5b2-0bce15370005 None admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:19.988185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2839060-3d8c-439b-b5b2-0bce15370005 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:19.988996 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2839060-3d8c-439b-b5b2-0bce15370005 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:19.990488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2839060-3d8c-439b-b5b2-0bce15370005 None admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:20.015906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 646/1306] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:14:19 2022] POST /identity/v3/endpoints => generated 331 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:14:29.739269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=30 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:30.010699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:15:37.724488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0e474-9f08-4839-93d6-73bce4325220 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:37.725702 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0e474-9f08-4839-93d6-73bce4325220 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:37.726204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0e474-9f08-4839-93d6-73bce4325220 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:37.733865 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 661/1307] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:15:37 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:15:37.736716 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e5217-e3d0-483d-a44f-258f32c53d4f None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:37.737363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e5217-e3d0-483d-a44f-258f32c53d4f None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:37.738139 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e5217-e3d0-483d-a44f-258f32c53d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:38.048450 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5e5e5217-e3d0-483d-a44f-258f32c53d4f None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:15:38.049792 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5e5e5217-e3d0-483d-a44f-258f32c53d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:38.283455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 647/1308] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 549 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:15:38.621396 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430db5c-8703-45a2-aade-bdcafa2456df None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:38.621899 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430db5c-8703-45a2-aade-bdcafa2456df None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:38.622052 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430db5c-8703-45a2-aade-bdcafa2456df None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:38.623963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 662/1309] 10.222.0.88 () {54 vars in 965 bytes} [Tue Aug 23 22:15:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:15:38.633683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b6b99-53bf-4f3e-8eda-54cb3b326af2 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:38.633833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b6b99-53bf-4f3e-8eda-54cb3b326af2 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:38.634092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b6b99-53bf-4f3e-8eda-54cb3b326af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:38.783240 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-f27b6b99-53bf-4f3e-8eda-54cb3b326af2 None None] MFA Rules not processed for user `6505100494d4496591407c48b48535ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:15:38.784578 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f27b6b99-53bf-4f3e-8eda-54cb3b326af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:38.821558 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:15:38.896893 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 648/1310] 10.222.0.88 () {58 vars in 1062 bytes} [Tue Aug 23 22:15:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:15:38.912677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430db5c-8703-45a2-aade-bdcafa2456df None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:15:38.916444 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d430db5c-8703-45a2-aade-bdcafa2456df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:39.095727 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430db5c-8703-45a2-aade-bdcafa2456df None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:15:39.188768 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9cec5-d1dd-444c-a7e5-8dd8b7f6e64c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:15:39.189418 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9cec5-d1dd-444c-a7e5-8dd8b7f6e64c service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:39.189703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9cec5-d1dd-444c-a7e5-8dd8b7f6e64c service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:39.189971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9cec5-d1dd-444c-a7e5-8dd8b7f6e64c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:39.192287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fae9cec5-d1dd-444c-a7e5-8dd8b7f6e64c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:39.260109 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:15:39.334975 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9cec5-d1dd-444c-a7e5-8dd8b7f6e64c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:15:39.336467 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9cec5-d1dd-444c-a7e5-8dd8b7f6e64c service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:15:39.365374 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 663/1311] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:15:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:15:42.541352 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79445be6-1506-474a-bb29-5f1e15c6ead1 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:42.542140 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79445be6-1506-474a-bb29-5f1e15c6ead1 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:42.542140 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79445be6-1506-474a-bb29-5f1e15c6ead1 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:42.543365 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 649/1312] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:15:42.550603 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667d39c-6048-457c-9eb4-34926ab0c6ee None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:42.551879 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667d39c-6048-457c-9eb4-34926ab0c6ee None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:42.552229 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667d39c-6048-457c-9eb4-34926ab0c6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:42.635050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-5667d39c-6048-457c-9eb4-34926ab0c6ee None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:15:42.636296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5667d39c-6048-457c-9eb4-34926ab0c6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:42.787201 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 664/1313] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:15:43.113442 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79445be6-1506-474a-bb29-5f1e15c6ead1 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:15:43.117574 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-79445be6-1506-474a-bb29-5f1e15c6ead1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:43.276654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79445be6-1506-474a-bb29-5f1e15c6ead1 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:15:43.391165 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:15:43.395297 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438d8e3-782e-4a84-b862-759a0ffa88bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:15:43.396802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438d8e3-782e-4a84-b862-759a0ffa88bb service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:43.396802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438d8e3-782e-4a84-b862-759a0ffa88bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:43.397208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438d8e3-782e-4a84-b862-759a0ffa88bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:43.399393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-2438d8e3-782e-4a84-b862-759a0ffa88bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:43.561163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438d8e3-782e-4a84-b862-759a0ffa88bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:15:43.562737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438d8e3-782e-4a84-b862-759a0ffa88bb service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:15:43.587054 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 650/1314] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:15:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:15:46.058952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c2f4c-0ac3-49d3-bb8e-e6d25eff18e7 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:46.059558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c2f4c-0ac3-49d3-bb8e-e6d25eff18e7 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:46.059558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c2f4c-0ac3-49d3-bb8e-e6d25eff18e7 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:46.060777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 665/1315] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:15:46.067584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e053f7-6775-461f-ba5a-703f9fb4ad3e None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:46.068192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e053f7-6775-461f-ba5a-703f9fb4ad3e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:46.068192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e053f7-6775-461f-ba5a-703f9fb4ad3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:46.143671 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-94e053f7-6775-461f-ba5a-703f9fb4ad3e None None] MFA Rules not processed for user `2a83b38dedc44dd89eb854ab6be2e94d`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:15:46.145601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-94e053f7-6775-461f-ba5a-703f9fb4ad3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:46.210749 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:15:46.264706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 651/1316] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:15:46 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:15:46.557460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c2f4c-0ac3-49d3-bb8e-e6d25eff18e7 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:15:46.559036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-691c2f4c-0ac3-49d3-bb8e-e6d25eff18e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:46.570141 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:15:46.684125 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c2f4c-0ac3-49d3-bb8e-e6d25eff18e7 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:15:46.783193 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026bddd-b030-47c3-8a6a-c750dc081160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:15:46.783801 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026bddd-b030-47c3-8a6a-c750dc081160 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:46.784079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026bddd-b030-47c3-8a6a-c750dc081160 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:46.784355 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026bddd-b030-47c3-8a6a-c750dc081160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:46.786050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b026bddd-b030-47c3-8a6a-c750dc081160 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:46.855011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:15:46.906331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026bddd-b030-47c3-8a6a-c750dc081160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:15:46.907800 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026bddd-b030-47c3-8a6a-c750dc081160 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:15:46.932379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 666/1317] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:15:46 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:15:49.676500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47109a91-9a73-4fb9-a8fc-291b5546f409 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:49.677363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47109a91-9a73-4fb9-a8fc-291b5546f409 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:49.677717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47109a91-9a73-4fb9-a8fc-291b5546f409 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:49.679460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 652/1318] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:15:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:15:49.687188 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9ecca-af8b-4554-984d-7df26a65fb1a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:49.687760 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9ecca-af8b-4554-984d-7df26a65fb1a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:49.688182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9ecca-af8b-4554-984d-7df26a65fb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:49.808642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-19c9ecca-af8b-4554-984d-7df26a65fb1a None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:15:49.810807 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-19c9ecca-af8b-4554-984d-7df26a65fb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:50.008753 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 667/1319] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:15:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:15:50.338574 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7726f-84de-4a59-a033-93e4008b7443 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:50.338862 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7726f-84de-4a59-a033-93e4008b7443 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:50.339213 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7726f-84de-4a59-a033-93e4008b7443 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:50.340653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 653/1320] 10.222.0.88 () {54 vars in 965 bytes} [Tue Aug 23 22:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:15:50.348675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f9ad2-a62b-4797-9da3-a4611e553968 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:50.349212 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f9ad2-a62b-4797-9da3-a4611e553968 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:50.349718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f9ad2-a62b-4797-9da3-a4611e553968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:50.391027 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:15:50.485602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b75f9ad2-a62b-4797-9da3-a4611e553968 None None] MFA Rules not processed for user `6505100494d4496591407c48b48535ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:15:50.487019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b75f9ad2-a62b-4797-9da3-a4611e553968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:50.591079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 668/1321] 10.222.0.88 () {58 vars in 1062 bytes} [Tue Aug 23 22:15:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:15:50.606249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7726f-84de-4a59-a033-93e4008b7443 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:15:50.608300 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-aaa7726f-84de-4a59-a033-93e4008b7443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:50.781541 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7726f-84de-4a59-a033-93e4008b7443 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:15:50.936015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f6305-22b6-4aee-ac92-5fef9471b8a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:15:50.937096 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f6305-22b6-4aee-ac92-5fef9471b8a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:50.937586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f6305-22b6-4aee-ac92-5fef9471b8a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:50.938084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f6305-22b6-4aee-ac92-5fef9471b8a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:50.941575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-229f6305-22b6-4aee-ac92-5fef9471b8a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:50.958109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:15:51.133491 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f6305-22b6-4aee-ac92-5fef9471b8a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:15:51.136235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f6305-22b6-4aee-ac92-5fef9471b8a5 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:15:51.172151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 654/1322] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:15:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:15:55.318598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82722d44-38f8-495f-a118-6ead712976a9 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:55.319149 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82722d44-38f8-495f-a118-6ead712976a9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:55.319269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82722d44-38f8-495f-a118-6ead712976a9 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:55.320579 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 669/1323] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:15:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:15:55.326259 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a209c-76ef-4168-9826-96b0b684a83e None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:55.326259 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a209c-76ef-4168-9826-96b0b684a83e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:55.327129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a209c-76ef-4168-9826-96b0b684a83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:55.429117 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-100a209c-76ef-4168-9826-96b0b684a83e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:15:55.431180 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-100a209c-76ef-4168-9826-96b0b684a83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:55.525246 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:15:55.577528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 655/1324] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:15:55.976989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82722d44-38f8-495f-a118-6ead712976a9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:15:55.978721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-82722d44-38f8-495f-a118-6ead712976a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:56.120261 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82722d44-38f8-495f-a118-6ead712976a9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:15:56.137238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:15:56.224513 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b79b6-44fb-4190-9688-aff19daeb399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:15:56.225606 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b79b6-44fb-4190-9688-aff19daeb399 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:56.225953 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b79b6-44fb-4190-9688-aff19daeb399 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:56.226287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b79b6-44fb-4190-9688-aff19daeb399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:56.228537 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4a0b79b6-44fb-4190-9688-aff19daeb399 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:56.425294 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b79b6-44fb-4190-9688-aff19daeb399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:15:56.428873 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b79b6-44fb-4190-9688-aff19daeb399 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:15:56.473576 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 670/1325] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:15:59.093660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce773d10-d592-43a9-a75f-c40b4a9ab0e4 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:59.094487 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce773d10-d592-43a9-a75f-c40b4a9ab0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:59.094487 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce773d10-d592-43a9-a75f-c40b4a9ab0e4 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:59.096361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 656/1326] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:15:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:15:59.106187 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349de600-60c8-46ae-8197-f09ca259a6c6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:15:59.106900 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349de600-60c8-46ae-8197-f09ca259a6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:15:59.107112 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349de600-60c8-46ae-8197-f09ca259a6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:15:59.202160 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:15:59.238838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-349de600-60c8-46ae-8197-f09ca259a6c6 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:15:59.240159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-349de600-60c8-46ae-8197-f09ca259a6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:59.409486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 671/1327] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:15:59.777678 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce773d10-d592-43a9-a75f-c40b4a9ab0e4 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:15:59.779555 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ce773d10-d592-43a9-a75f-c40b4a9ab0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:15:59.992079 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce773d10-d592-43a9-a75f-c40b4a9ab0e4 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:16:00.136496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a24ba-2053-4d04-8208-c0f0ac7b47fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:16:00.137823 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a24ba-2053-4d04-8208-c0f0ac7b47fc service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:00.137823 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a24ba-2053-4d04-8208-c0f0ac7b47fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:00.138152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a24ba-2053-4d04-8208-c0f0ac7b47fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:00.141325 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c28a24ba-2053-4d04-8208-c0f0ac7b47fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:00.167567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:00.344091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a24ba-2053-4d04-8208-c0f0ac7b47fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:16:00.347088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a24ba-2053-4d04-8208-c0f0ac7b47fc service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:16:00.386231 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 657/1328] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:16:02.934943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1686a-626d-4cb6-9366-573626a898c2 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:02.935593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1686a-626d-4cb6-9366-573626a898c2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:02.935593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1686a-626d-4cb6-9366-573626a898c2 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:02.936896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 672/1329] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:16:02.943647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e76a3-8b41-47f0-8047-077cc1b9acdf None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:02.944238 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e76a3-8b41-47f0-8047-077cc1b9acdf None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:02.944238 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e76a3-8b41-47f0-8047-077cc1b9acdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:03.046075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-4d0e76a3-8b41-47f0-8047-077cc1b9acdf None None] MFA Rules not processed for user `2a83b38dedc44dd89eb854ab6be2e94d`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:03.048073 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4d0e76a3-8b41-47f0-8047-077cc1b9acdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:03.150984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:03.190336 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 658/1330] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:16:02 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:16:03.509125 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1686a-626d-4cb6-9366-573626a898c2 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:16:03.510657 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1bc1686a-626d-4cb6-9366-573626a898c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:03.652632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1686a-626d-4cb6-9366-573626a898c2 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:16:03.678774 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:03.776483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea83627-6a08-42fe-8422-414ee0f55024 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:16:03.777140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea83627-6a08-42fe-8422-414ee0f55024 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:03.777424 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea83627-6a08-42fe-8422-414ee0f55024 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:03.777710 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea83627-6a08-42fe-8422-414ee0f55024 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:03.779424 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5ea83627-6a08-42fe-8422-414ee0f55024 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:03.928251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea83627-6a08-42fe-8422-414ee0f55024 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:16:03.929894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea83627-6a08-42fe-8422-414ee0f55024 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:16:03.951689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 673/1331] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:16:03 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:16:06.575956 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ef3d9-458a-4769-930c-155e73f0979b None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:06.576681 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ef3d9-458a-4769-930c-155e73f0979b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:06.576681 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ef3d9-458a-4769-930c-155e73f0979b None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:06.577915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 659/1332] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:16:06.584967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be77cc-3faa-41be-9dc0-8d8e64627940 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:06.585574 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be77cc-3faa-41be-9dc0-8d8e64627940 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:06.585574 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be77cc-3faa-41be-9dc0-8d8e64627940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:06.667496 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:06.670876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f0be77cc-3faa-41be-9dc0-8d8e64627940 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:06.672736 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f0be77cc-3faa-41be-9dc0-8d8e64627940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:06.789230 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 674/1333] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:16:07.089558 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ef3d9-458a-4769-930c-155e73f0979b None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:16:07.092378 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8d8ef3d9-458a-4769-930c-155e73f0979b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:07.237894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ef3d9-458a-4769-930c-155e73f0979b None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:16:07.331425 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6f617-5286-422e-9afd-1df8d628d128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:16:07.332031 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6f617-5286-422e-9afd-1df8d628d128 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:07.332460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6f617-5286-422e-9afd-1df8d628d128 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:07.333176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6f617-5286-422e-9afd-1df8d628d128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:07.334860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-9bd6f617-5286-422e-9afd-1df8d628d128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:07.366915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:07.469165 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6f617-5286-422e-9afd-1df8d628d128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:16:07.470887 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6f617-5286-422e-9afd-1df8d628d128 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:16:07.505484 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 660/1334] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:16:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:16:09.605762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259ea35-ed1e-4e9a-9dde-adfaf162e7b6 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:09.606481 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259ea35-ed1e-4e9a-9dde-adfaf162e7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:09.606481 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259ea35-ed1e-4e9a-9dde-adfaf162e7b6 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:09.607699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 675/1335] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:16:09.614381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fee9e-94af-4689-b506-49dd2a832f3c None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:09.615138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fee9e-94af-4689-b506-49dd2a832f3c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:09.615297 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fee9e-94af-4689-b506-49dd2a832f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:09.688493 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-0b2fee9e-94af-4689-b506-49dd2a832f3c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:09.689757 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0b2fee9e-94af-4689-b506-49dd2a832f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:09.799981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:09.826123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 661/1336] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:16:10.116427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259ea35-ed1e-4e9a-9dde-adfaf162e7b6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:16:10.118406 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9259ea35-ed1e-4e9a-9dde-adfaf162e7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:10.229870 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259ea35-ed1e-4e9a-9dde-adfaf162e7b6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:16:10.262047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:10.316108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983b511-f588-41bd-8712-7fdf2e9a873d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:16:10.316786 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983b511-f588-41bd-8712-7fdf2e9a873d service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:10.317292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983b511-f588-41bd-8712-7fdf2e9a873d service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:10.317774 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983b511-f588-41bd-8712-7fdf2e9a873d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:10.319696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0983b511-f588-41bd-8712-7fdf2e9a873d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:10.518076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983b511-f588-41bd-8712-7fdf2e9a873d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:16:10.519641 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983b511-f588-41bd-8712-7fdf2e9a873d service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:16:10.548443 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 676/1337] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:16:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:16:13.035741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3a72f-9e19-432f-8246-ad29bd3c9815 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:13.036563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3a72f-9e19-432f-8246-ad29bd3c9815 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:13.036563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3a72f-9e19-432f-8246-ad29bd3c9815 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:13.037606 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 662/1338] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:16:13.043590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a0a3b-fe5a-4920-ae99-cdcddbc26011 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:13.044133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a0a3b-fe5a-4920-ae99-cdcddbc26011 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:13.044481 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a0a3b-fe5a-4920-ae99-cdcddbc26011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:13.121881 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d12a0a3b-fe5a-4920-ae99-cdcddbc26011 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:13.123129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d12a0a3b-fe5a-4920-ae99-cdcddbc26011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:13.133076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:13.235804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 677/1339] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:16:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:16:13.546149 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3a72f-9e19-432f-8246-ad29bd3c9815 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:16:13.547841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ced3a72f-9e19-432f-8246-ad29bd3c9815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:13.668452 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3a72f-9e19-432f-8246-ad29bd3c9815 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:16:13.751233 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbd130-b0f0-4716-be0c-b71a22a9e50a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:16:13.751857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbd130-b0f0-4716-be0c-b71a22a9e50a service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:13.752130 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbd130-b0f0-4716-be0c-b71a22a9e50a service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:13.752404 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbd130-b0f0-4716-be0c-b71a22a9e50a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:13.754083 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-effbd130-b0f0-4716-be0c-b71a22a9e50a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:13.794120 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:13.881082 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbd130-b0f0-4716-be0c-b71a22a9e50a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:16:13.882561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbd130-b0f0-4716-be0c-b71a22a9e50a service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:16:13.910203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 663/1340] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:16:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:16:18.213962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f106270-2ea0-4c62-81c1-0fb56effbf42 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:18.214851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f106270-2ea0-4c62-81c1-0fb56effbf42 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:18.214851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f106270-2ea0-4c62-81c1-0fb56effbf42 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:18.216433 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 678/1341] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:16:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:16:18.222814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2efbc-a238-4b04-a6be-e602c5b6cdd7 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:18.224435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2efbc-a238-4b04-a6be-e602c5b6cdd7 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:18.224435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2efbc-a238-4b04-a6be-e602c5b6cdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:18.310603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-24e2efbc-a238-4b04-a6be-e602c5b6cdd7 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:18.311885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-24e2efbc-a238-4b04-a6be-e602c5b6cdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:18.424608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:18.441364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 664/1342] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:16:18.748513 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f106270-2ea0-4c62-81c1-0fb56effbf42 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:16:18.751658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9f106270-2ea0-4c62-81c1-0fb56effbf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:18.916446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f106270-2ea0-4c62-81c1-0fb56effbf42 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:16:18.961125 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:19.006482 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc6710-c656-4e20-ae47-f08033aff30d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:16:19.007019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc6710-c656-4e20-ae47-f08033aff30d service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:19.007326 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc6710-c656-4e20-ae47-f08033aff30d service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:19.007647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc6710-c656-4e20-ae47-f08033aff30d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:19.009503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fcbc6710-c656-4e20-ae47-f08033aff30d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:19.152734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc6710-c656-4e20-ae47-f08033aff30d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:16:19.154306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc6710-c656-4e20-ae47-f08033aff30d service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:16:19.178888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 679/1343] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:16:22.372802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689a960-da3b-43fe-8d34-fbc375488730 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:22.373442 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689a960-da3b-43fe-8d34-fbc375488730 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:22.373824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689a960-da3b-43fe-8d34-fbc375488730 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:22.376661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 665/1344] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:16:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:16:22.384686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb636a-d142-4712-8768-c4ce10ce194a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:22.385385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb636a-d142-4712-8768-c4ce10ce194a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:22.385385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb636a-d142-4712-8768-c4ce10ce194a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:22.469858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-63bb636a-d142-4712-8768-c4ce10ce194a None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:22.471286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-63bb636a-d142-4712-8768-c4ce10ce194a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:22.492383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:22.617723 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 680/1345] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:16:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:16:23.146980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689a960-da3b-43fe-8d34-fbc375488730 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:16:23.150233 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-7689a960-da3b-43fe-8d34-fbc375488730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:23.326017 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689a960-da3b-43fe-8d34-fbc375488730 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:16:23.445186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc4ba0-4c3a-42b8-9d28-ea99c9dc1da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:16:23.446212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc4ba0-4c3a-42b8-9d28-ea99c9dc1da0 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:23.446212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc4ba0-4c3a-42b8-9d28-ea99c9dc1da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:23.446523 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc4ba0-4c3a-42b8-9d28-ea99c9dc1da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:23.448034 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-21dc4ba0-4c3a-42b8-9d28-ea99c9dc1da0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:23.513118 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:23.646176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc4ba0-4c3a-42b8-9d28-ea99c9dc1da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:16:23.649245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc4ba0-4c3a-42b8-9d28-ea99c9dc1da0 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:16:23.676653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 666/1346] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:16:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:16:28.474042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1c09c-8d50-4412-be12-423e19869b87 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:28.474836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1c09c-8d50-4412-be12-423e19869b87 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:28.474836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1c09c-8d50-4412-be12-423e19869b87 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:28.476429 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 681/1347] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:16:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:16:28.482185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67113e-59b3-4e31-a2d3-211f54f4a04c None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:28.482738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67113e-59b3-4e31-a2d3-211f54f4a04c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:28.482956 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67113e-59b3-4e31-a2d3-211f54f4a04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:28.627331 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-eb67113e-59b3-4e31-a2d3-211f54f4a04c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:28.630386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-eb67113e-59b3-4e31-a2d3-211f54f4a04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:28.804223 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:28.813820 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 667/1348] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:16:29.347627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1c09c-8d50-4412-be12-423e19869b87 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:16:29.349962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4bb1c09c-8d50-4412-be12-423e19869b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:29.523917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1c09c-8d50-4412-be12-423e19869b87 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:16:29.584645 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:29.620569 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3fd8c-32f4-4ecb-bf47-0c5a6a3df8bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:16:29.621322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3fd8c-32f4-4ecb-bf47-0c5a6a3df8bd service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:29.621609 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3fd8c-32f4-4ecb-bf47-0c5a6a3df8bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:29.621939 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3fd8c-32f4-4ecb-bf47-0c5a6a3df8bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:29.623888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-47d3fd8c-32f4-4ecb-bf47-0c5a6a3df8bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:29.764650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3fd8c-32f4-4ecb-bf47-0c5a6a3df8bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:16:29.764940 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3fd8c-32f4-4ecb-bf47-0c5a6a3df8bd service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:16:29.791478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 682/1349] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:16:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:16:35.498779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0b7d5-0184-485b-9979-af31f8db8345 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:35.499615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0b7d5-0184-485b-9979-af31f8db8345 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:35.499615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0b7d5-0184-485b-9979-af31f8db8345 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:35.501040 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 668/1350] 10.222.0.88 () {54 vars in 921 bytes} [Tue Aug 23 22:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:16:35.511236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04453b89-9f65-4b90-8d5a-86ab02ff9fa6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:35.512808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04453b89-9f65-4b90-8d5a-86ab02ff9fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:35.513607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04453b89-9f65-4b90-8d5a-86ab02ff9fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:35.670504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-04453b89-9f65-4b90-8d5a-86ab02ff9fa6 None None] MFA Rules not processed for user `e642dd7a07ca4afaa40c2769299d8bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:35.672833 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-04453b89-9f65-4b90-8d5a-86ab02ff9fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:35.713596 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:35.882007 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 683/1351] 10.222.0.88 () {58 vars in 1018 bytes} [Tue Aug 23 22:16:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:16:41.278149 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b364e6-d934-4526-b8f3-8ae1662b9d74 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:41.278711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b364e6-d934-4526-b8f3-8ae1662b9d74 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:41.278711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b364e6-d934-4526-b8f3-8ae1662b9d74 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:41.280041 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 669/1352] 10.222.0.88 () {54 vars in 921 bytes} [Tue Aug 23 22:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:16:41.285671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d04b54-e28b-4357-98e1-30ed6b793d46 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:41.286731 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d04b54-e28b-4357-98e1-30ed6b793d46 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:41.286850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d04b54-e28b-4357-98e1-30ed6b793d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:41.392675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-37d04b54-e28b-4357-98e1-30ed6b793d46 None None] MFA Rules not processed for user `e642dd7a07ca4afaa40c2769299d8bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:41.393923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-37d04b54-e28b-4357-98e1-30ed6b793d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:41.491088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:41.541722 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 684/1353] 10.222.0.88 () {58 vars in 1018 bytes} [Tue Aug 23 22:16:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:16:42.942682 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7b17d-f2e6-4178-819b-d37b9ad9812c None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:42.943515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7b17d-f2e6-4178-819b-d37b9ad9812c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:42.943515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7b17d-f2e6-4178-819b-d37b9ad9812c None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:42.944874 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 670/1354] 10.222.0.88 () {54 vars in 921 bytes} [Tue Aug 23 22:16:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:16:42.950405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac5761-7fd6-4177-8687-eb2591a93fe6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:42.950866 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac5761-7fd6-4177-8687-eb2591a93fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:42.951053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac5761-7fd6-4177-8687-eb2591a93fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:43.044423 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-67ac5761-7fd6-4177-8687-eb2591a93fe6 None None] MFA Rules not processed for user `e642dd7a07ca4afaa40c2769299d8bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:43.045640 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-67ac5761-7fd6-4177-8687-eb2591a93fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:43.178153 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 685/1355] 10.222.0.88 () {58 vars in 1018 bytes} [Tue Aug 23 22:16:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:16:46.734953 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666953c-f170-4b08-bcee-8ac898264a12 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:46.735835 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666953c-f170-4b08-bcee-8ac898264a12 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:46.736007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666953c-f170-4b08-bcee-8ac898264a12 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:46.737610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 671/1356] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:16:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:16:46.745505 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1dda0-e4b7-4f62-9e14-f7e7054897f5 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:46.746240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1dda0-e4b7-4f62-9e14-f7e7054897f5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:46.746240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1dda0-e4b7-4f62-9e14-f7e7054897f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:46.806883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:46.857096 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-61e1dda0-e4b7-4f62-9e14-f7e7054897f5 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:46.862740 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-61e1dda0-e4b7-4f62-9e14-f7e7054897f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:47.043999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 686/1357] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:16:47.559834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a50cb9-6fd0-4b49-ba45-a3f773b16c5a None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:47.560337 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a50cb9-6fd0-4b49-ba45-a3f773b16c5a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:47.560825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a50cb9-6fd0-4b49-ba45-a3f773b16c5a None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:47.563374 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 672/1358] 10.222.0.88 () {54 vars in 962 bytes} [Tue Aug 23 22:16:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:16:47.578752 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed1450-cd6b-44b1-8e7b-4671f4985d6f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:47.579592 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed1450-cd6b-44b1-8e7b-4671f4985d6f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:47.580282 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed1450-cd6b-44b1-8e7b-4671f4985d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:47.724437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-54ed1450-cd6b-44b1-8e7b-4671f4985d6f None None] MFA Rules not processed for user `e3517b3b077a4a1fa76163657b46930c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:47.725792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-54ed1450-cd6b-44b1-8e7b-4671f4985d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:47.782728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:47.875663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 687/1359] 10.222.0.88 () {58 vars in 1059 bytes} [Tue Aug 23 22:16:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:16:47.884278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a50cb9-6fd0-4b49-ba45-a3f773b16c5a None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:16:47.885860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a7a50cb9-6fd0-4b49-ba45-a3f773b16c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:48.061861 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a50cb9-6fd0-4b49-ba45-a3f773b16c5a None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:16:48.184707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:48.267067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ff646-11e7-4336-bd73-adb63965183d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:16:48.268021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ff646-11e7-4336-bd73-adb63965183d service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:48.268465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ff646-11e7-4336-bd73-adb63965183d service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:48.268914 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ff646-11e7-4336-bd73-adb63965183d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:48.272526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-2f6ff646-11e7-4336-bd73-adb63965183d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:48.452397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ff646-11e7-4336-bd73-adb63965183d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:16:48.454854 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ff646-11e7-4336-bd73-adb63965183d service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:16:48.490571 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 673/1360] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:16:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:16:51.543891 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5da14-116d-4a04-8e1b-a2482c6aeb0f None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:51.545190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5da14-116d-4a04-8e1b-a2482c6aeb0f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:51.545719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5da14-116d-4a04-8e1b-a2482c6aeb0f None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:51.548832 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 688/1361] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:16:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:16:51.556558 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5143eeb-21ea-4e2c-8fb4-3eb757d4645c None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:51.557674 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5143eeb-21ea-4e2c-8fb4-3eb757d4645c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:51.557674 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5143eeb-21ea-4e2c-8fb4-3eb757d4645c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:51.672515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-c5143eeb-21ea-4e2c-8fb4-3eb757d4645c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:51.675050 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c5143eeb-21ea-4e2c-8fb4-3eb757d4645c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:51.787394 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:51.935204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 674/1362] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:16:52.443988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5da14-116d-4a04-8e1b-a2482c6aeb0f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:16:52.447298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-98f5da14-116d-4a04-8e1b-a2482c6aeb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:52.677358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5da14-116d-4a04-8e1b-a2482c6aeb0f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:16:52.689819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:52.819580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f237ae0-ae1b-4ce0-9cf7-3b0f02e86557 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:16:52.820269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f237ae0-ae1b-4ce0-9cf7-3b0f02e86557 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:52.820594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f237ae0-ae1b-4ce0-9cf7-3b0f02e86557 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:52.821193 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f237ae0-ae1b-4ce0-9cf7-3b0f02e86557 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:52.823577 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4f237ae0-ae1b-4ce0-9cf7-3b0f02e86557 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:52.959729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f237ae0-ae1b-4ce0-9cf7-3b0f02e86557 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:16:52.965218 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f237ae0-ae1b-4ce0-9cf7-3b0f02e86557 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:16:53.002847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 689/1363] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:16:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:16:56.049374 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcd11b-5144-4218-9ea2-8d5d1584ea09 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:56.050805 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcd11b-5144-4218-9ea2-8d5d1584ea09 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:56.051572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcd11b-5144-4218-9ea2-8d5d1584ea09 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:56.054295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 675/1364] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:16:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:16:56.066354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5ab42-8b5f-4686-9ea2-14e6c67329b3 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:56.069975 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5ab42-8b5f-4686-9ea2-14e6c67329b3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:56.071014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5ab42-8b5f-4686-9ea2-14e6c67329b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:56.101308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:56.189389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f6b5ab42-8b5f-4686-9ea2-14e6c67329b3 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:56.190990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f6b5ab42-8b5f-4686-9ea2-14e6c67329b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:56.386648 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 690/1365] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:16:56.974544 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f320b4-2ce6-42a0-a918-435af1bb3f8f None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:56.975232 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f320b4-2ce6-42a0-a918-435af1bb3f8f None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:56.975841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f320b4-2ce6-42a0-a918-435af1bb3f8f None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:56.978701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 676/1366] 10.222.0.88 () {54 vars in 962 bytes} [Tue Aug 23 22:16:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:16:56.988557 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5e217-b933-44b9-bc59-541f4547aa5a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:56.989584 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5e217-b933-44b9-bc59-541f4547aa5a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:56.990049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5e217-b933-44b9-bc59-541f4547aa5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:57.125498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d5e5e217-b933-44b9-bc59-541f4547aa5a None None] MFA Rules not processed for user `e3517b3b077a4a1fa76163657b46930c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:57.126962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d5e5e217-b933-44b9-bc59-541f4547aa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:57.214454 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:57.421403 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 691/1367] 10.222.0.88 () {58 vars in 1059 bytes} [Tue Aug 23 22:16:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:16:57.432985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f320b4-2ce6-42a0-a918-435af1bb3f8f None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:16:57.435125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e1f320b4-2ce6-42a0-a918-435af1bb3f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:57.722100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f320b4-2ce6-42a0-a918-435af1bb3f8f None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:16:57.758798 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:57.987873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02ebb7-e2b9-4973-9568-1420c98741b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:16:57.989369 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02ebb7-e2b9-4973-9568-1420c98741b1 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:57.990064 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02ebb7-e2b9-4973-9568-1420c98741b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:57.990795 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02ebb7-e2b9-4973-9568-1420c98741b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:57.994191 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8f02ebb7-e2b9-4973-9568-1420c98741b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:58.144358 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02ebb7-e2b9-4973-9568-1420c98741b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:16:58.145661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02ebb7-e2b9-4973-9568-1420c98741b1 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:16:58.170916 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 677/1368] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:16:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:16:58.354446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1b490-aa10-48dd-add5-bd9e836009df None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:58.355281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1b490-aa10-48dd-add5-bd9e836009df None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:58.355281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1b490-aa10-48dd-add5-bd9e836009df None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:58.357041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 692/1369] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:16:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:16:58.364748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df203672-2c4c-41e5-86b3-079eff8588d0 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:58.365404 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df203672-2c4c-41e5-86b3-079eff8588d0 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:58.365892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df203672-2c4c-41e5-86b3-079eff8588d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:58.410397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:58.482395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-df203672-2c4c-41e5-86b3-079eff8588d0 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:58.484529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-df203672-2c4c-41e5-86b3-079eff8588d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:58.702032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 678/1370] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:16:59.387840 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b11248-153a-411e-b29a-4234960850cc None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:59.388566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b11248-153a-411e-b29a-4234960850cc None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:59.388683 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b11248-153a-411e-b29a-4234960850cc None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:59.397015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 693/1371] 10.222.0.88 () {54 vars in 964 bytes} [Tue Aug 23 22:16:59 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:16:59.405793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9719cd-0ae4-425c-90f9-bbd3bb95736e None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:16:59.406409 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9719cd-0ae4-425c-90f9-bbd3bb95736e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:16:59.406926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9719cd-0ae4-425c-90f9-bbd3bb95736e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:16:59.545474 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-8e9719cd-0ae4-425c-90f9-bbd3bb95736e None None] MFA Rules not processed for user `7d9ebd88431d46348089bc195491fca5`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:16:59.546871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8e9719cd-0ae4-425c-90f9-bbd3bb95736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:59.621583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:16:59.680568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 679/1372] 10.222.0.88 () {58 vars in 1061 bytes} [Tue Aug 23 22:16:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:16:59.695042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b11248-153a-411e-b29a-4234960850cc None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:16:59.698112 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-55b11248-153a-411e-b29a-4234960850cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:16:59.928834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b11248-153a-411e-b29a-4234960850cc None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:00.025944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:00.102962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce288bb-6587-4b1d-94d3-5aa91706c732 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:00.103958 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce288bb-6587-4b1d-94d3-5aa91706c732 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:00.104453 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce288bb-6587-4b1d-94d3-5aa91706c732 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:00.105422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce288bb-6587-4b1d-94d3-5aa91706c732 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:00.110882 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-cce288bb-6587-4b1d-94d3-5aa91706c732 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:00.331999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce288bb-6587-4b1d-94d3-5aa91706c732 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:00.334775 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce288bb-6587-4b1d-94d3-5aa91706c732 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:00.366771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 694/1373] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:16:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 673 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:17:00.768321 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b19795-06bd-4ab0-916f-067835e9a4f9 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:00.769174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b19795-06bd-4ab0-916f-067835e9a4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:00.769174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b19795-06bd-4ab0-916f-067835e9a4f9 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:00.770829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 680/1374] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:17:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:17:00.777662 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b563c-03cb-40ee-a1de-c4e413ba3c2c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:00.778375 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b563c-03cb-40ee-a1de-c4e413ba3c2c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:00.778375 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b563c-03cb-40ee-a1de-c4e413ba3c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:00.900080 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-697b563c-03cb-40ee-a1de-c4e413ba3c2c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:00.901707 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-697b563c-03cb-40ee-a1de-c4e413ba3c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:00.939119 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:01.064925 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 695/1375] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:01.402811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b19795-06bd-4ab0-916f-067835e9a4f9 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:01.405333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-83b19795-06bd-4ab0-916f-067835e9a4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:01.726030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b19795-06bd-4ab0-916f-067835e9a4f9 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:01.824138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:01.921626 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615d274-f8ff-45f9-84f6-56d8faef1ac8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:01.922144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615d274-f8ff-45f9-84f6-56d8faef1ac8 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:01.922439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615d274-f8ff-45f9-84f6-56d8faef1ac8 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:01.922787 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615d274-f8ff-45f9-84f6-56d8faef1ac8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:01.924764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-2615d274-f8ff-45f9-84f6-56d8faef1ac8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:02.088515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615d274-f8ff-45f9-84f6-56d8faef1ac8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:02.092221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615d274-f8ff-45f9-84f6-56d8faef1ac8 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:02.133614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 681/1376] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 731 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:17:04.549488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4657d-35a6-41a2-996d-5d00e9d70c95 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:04.550324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4657d-35a6-41a2-996d-5d00e9d70c95 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:04.550505 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4657d-35a6-41a2-996d-5d00e9d70c95 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:04.553368 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 696/1377] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:17:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:17:04.560665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ed6cf-7d52-4a9c-93c9-68776e3b1552 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:04.561370 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ed6cf-7d52-4a9c-93c9-68776e3b1552 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:04.561540 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ed6cf-7d52-4a9c-93c9-68776e3b1552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:04.683356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-ac1ed6cf-7d52-4a9c-93c9-68776e3b1552 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:04.684920 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ac1ed6cf-7d52-4a9c-93c9-68776e3b1552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:04.694118 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:04.888918 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 682/1378] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:17:05.239015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4657d-35a6-41a2-996d-5d00e9d70c95 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:05.240828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c0a4657d-35a6-41a2-996d-5d00e9d70c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:05.469994 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4657d-35a6-41a2-996d-5d00e9d70c95 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:05.477334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:05.644847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98803fbc-1a5d-4e2f-8293-dd926b1a7162 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:05.646931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98803fbc-1a5d-4e2f-8293-dd926b1a7162 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:05.646931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98803fbc-1a5d-4e2f-8293-dd926b1a7162 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:05.647463 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98803fbc-1a5d-4e2f-8293-dd926b1a7162 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:05.649492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-98803fbc-1a5d-4e2f-8293-dd926b1a7162 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:05.762319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98803fbc-1a5d-4e2f-8293-dd926b1a7162 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:05.763381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98803fbc-1a5d-4e2f-8293-dd926b1a7162 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:05.788214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 697/1379] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:17:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:17:08.146590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd76467-200a-4a7b-a550-f84b0f727676 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:08.147339 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd76467-200a-4a7b-a550-f84b0f727676 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:08.147742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd76467-200a-4a7b-a550-f84b0f727676 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:08.150040 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 683/1380] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:17:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:17:08.157703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa38640-e9c9-4524-bbe2-d3ede18057cb None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:08.158376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa38640-e9c9-4524-bbe2-d3ede18057cb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:08.158376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa38640-e9c9-4524-bbe2-d3ede18057cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:08.173029 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:08.257794 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-caa38640-e9c9-4524-bbe2-d3ede18057cb None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:08.260094 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-caa38640-e9c9-4524-bbe2-d3ede18057cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:08.459910 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 698/1381] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:17:08.843065 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd76467-200a-4a7b-a550-f84b0f727676 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:08.845697 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5cd76467-200a-4a7b-a550-f84b0f727676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:09.104340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:09.123396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd76467-200a-4a7b-a550-f84b0f727676 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:09.279700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721f1ef-a1e5-42f8-b841-a6ee622f97b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:09.280439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721f1ef-a1e5-42f8-b841-a6ee622f97b2 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:09.280688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721f1ef-a1e5-42f8-b841-a6ee622f97b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:09.280960 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721f1ef-a1e5-42f8-b841-a6ee622f97b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:09.282821 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5721f1ef-a1e5-42f8-b841-a6ee622f97b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:09.444833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721f1ef-a1e5-42f8-b841-a6ee622f97b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:09.447562 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721f1ef-a1e5-42f8-b841-a6ee622f97b2 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:09.468265 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:09.485644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 684/1382] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:17:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:17:11.947141 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c47fe1-4a4e-4b13-aac5-43e9c001cd9a None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:11.947769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c47fe1-4a4e-4b13-aac5-43e9c001cd9a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:11.947769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c47fe1-4a4e-4b13-aac5-43e9c001cd9a None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:11.949077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 699/1383] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:17:11.955130 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1fa5a-a18c-4c26-955b-7dae7c19cfc0 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:11.955800 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1fa5a-a18c-4c26-955b-7dae7c19cfc0 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:11.955800 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1fa5a-a18c-4c26-955b-7dae7c19cfc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:12.052546 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-eee1fa5a-a18c-4c26-955b-7dae7c19cfc0 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:12.054315 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-eee1fa5a-a18c-4c26-955b-7dae7c19cfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:12.202581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 685/1384] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:12.526687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c47fe1-4a4e-4b13-aac5-43e9c001cd9a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:12.529458 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f1c47fe1-4a4e-4b13-aac5-43e9c001cd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:12.673641 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:12.778412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c47fe1-4a4e-4b13-aac5-43e9c001cd9a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:12.986844 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84dc876-b0de-4cbb-a8c6-ab1307f6902f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:12.987575 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84dc876-b0de-4cbb-a8c6-ab1307f6902f service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:12.987933 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84dc876-b0de-4cbb-a8c6-ab1307f6902f service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:12.988269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84dc876-b0de-4cbb-a8c6-ab1307f6902f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:12.990459 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b84dc876-b0de-4cbb-a8c6-ab1307f6902f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:13.086452 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:13.164928 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84dc876-b0de-4cbb-a8c6-ab1307f6902f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:13.168967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84dc876-b0de-4cbb-a8c6-ab1307f6902f service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:13.203932 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 700/1385] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:17:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 678 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:17:15.566353 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606957e-b134-44b5-9550-82e7ca944d4b None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:15.567625 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606957e-b134-44b5-9550-82e7ca944d4b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:15.567625 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606957e-b134-44b5-9550-82e7ca944d4b None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:15.569083 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 686/1386] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:17:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:17:15.576456 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f4810-8d20-4218-b88b-fff2336a9a2e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:15.577740 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f4810-8d20-4218-b88b-fff2336a9a2e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:15.577740 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f4810-8d20-4218-b88b-fff2336a9a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:15.678957 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0a9f4810-8d20-4218-b88b-fff2336a9a2e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:15.680282 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0a9f4810-8d20-4218-b88b-fff2336a9a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:15.826211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 701/1387] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:16.168229 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606957e-b134-44b5-9550-82e7ca944d4b None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:16.170013 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6606957e-b134-44b5-9550-82e7ca944d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:16.275586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:16.379977 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606957e-b134-44b5-9550-82e7ca944d4b None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:16.554222 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe08932-10c1-4ecc-9b3c-a8be3a3a1047 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:16.554960 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe08932-10c1-4ecc-9b3c-a8be3a3a1047 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:16.555375 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe08932-10c1-4ecc-9b3c-a8be3a3a1047 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:16.555796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe08932-10c1-4ecc-9b3c-a8be3a3a1047 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:16.558024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-dfe08932-10c1-4ecc-9b3c-a8be3a3a1047 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:16.623411 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:16.707421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe08932-10c1-4ecc-9b3c-a8be3a3a1047 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:16.709970 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe08932-10c1-4ecc-9b3c-a8be3a3a1047 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:16.734803 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 687/1388] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:17:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:17:18.381184 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d9c06-01fc-48e0-9511-5d7bc58c3e4d None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:18.381184 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d9c06-01fc-48e0-9511-5d7bc58c3e4d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:18.382614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d9c06-01fc-48e0-9511-5d7bc58c3e4d None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:18.383101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 702/1389] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:17:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:17:18.389441 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120a06c-0ffb-4d25-a0ff-d0dce52d59b6 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:18.390227 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120a06c-0ffb-4d25-a0ff-d0dce52d59b6 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:18.390436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120a06c-0ffb-4d25-a0ff-d0dce52d59b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:18.491296 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-2120a06c-0ffb-4d25-a0ff-d0dce52d59b6 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:18.492570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-2120a06c-0ffb-4d25-a0ff-d0dce52d59b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:18.671119 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 688/1390] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:19.030488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06402d7-1a12-4d73-bc5e-261d55d58cf2 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:19.030914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06402d7-1a12-4d73-bc5e-261d55d58cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:19.031281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06402d7-1a12-4d73-bc5e-261d55d58cf2 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:19.034473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 703/1391] 10.222.0.88 () {54 vars in 967 bytes} [Tue Aug 23 22:17:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:17:19.047117 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd48bf9-39c4-4b0c-aea4-90f295ba4e33 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:19.047610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd48bf9-39c4-4b0c-aea4-90f295ba4e33 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:19.048108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd48bf9-39c4-4b0c-aea4-90f295ba4e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:19.091269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:19.094290 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b611c-d521-45b2-990e-99cc709732ee None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:19.094290 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b611c-d521-45b2-990e-99cc709732ee None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:19.094629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b611c-d521-45b2-990e-99cc709732ee None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:19.095768 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 704/1392] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:17:19.106379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f390ff5-3a37-4ac3-9951-a57e4b97286a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:19.106875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f390ff5-3a37-4ac3-9951-a57e4b97286a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:19.107363 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f390ff5-3a37-4ac3-9951-a57e4b97286a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:19.168410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:19.172345 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-fbd48bf9-39c4-4b0c-aea4-90f295ba4e33 None None] MFA Rules not processed for user `2211e237fe554a82baf129e440a0fb43`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:19.173817 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-fbd48bf9-39c4-4b0c-aea4-90f295ba4e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:19.210362 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-5f390ff5-3a37-4ac3-9951-a57e4b97286a None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:19.211617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5f390ff5-3a37-4ac3-9951-a57e4b97286a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:19.286865 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 689/1393] 10.222.0.88 () {58 vars in 1064 bytes} [Tue Aug 23 22:17:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:19.294856 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd48bf9-39c4-4b0c-aea4-90f295ba4e33 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:19.305182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd48bf9-39c4-4b0c-aea4-90f295ba4e33 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:19.356688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 705/1394] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:19.435601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50608a-1730-4382-9340-58cc80049907 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2211e237fe554a82baf129e440a0fb43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:19.436338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50608a-1730-4382-9340-58cc80049907 service glance] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:19.436752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50608a-1730-4382-9340-58cc80049907 service glance] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:19.437250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50608a-1730-4382-9340-58cc80049907 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:19.440412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4a50608a-1730-4382-9340-58cc80049907 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:19.486231 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:19.608889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50608a-1730-4382-9340-58cc80049907 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:19.611058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50608a-1730-4382-9340-58cc80049907 service glance] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:19.649883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 690/1395] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:17:19.685771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f390ff5-3a37-4ac3-9951-a57e4b97286a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:19.687458 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5f390ff5-3a37-4ac3-9951-a57e4b97286a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:19.819859 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:19.831804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f390ff5-3a37-4ac3-9951-a57e4b97286a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:20.006263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c2d1e-eab8-48a3-828c-6862abe1b648 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:20.007056 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c2d1e-eab8-48a3-828c-6862abe1b648 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:20.007467 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c2d1e-eab8-48a3-828c-6862abe1b648 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:20.007780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c2d1e-eab8-48a3-828c-6862abe1b648 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:20.009690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-323c2d1e-eab8-48a3-828c-6862abe1b648 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:20.175691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c2d1e-eab8-48a3-828c-6862abe1b648 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:20.179707 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c2d1e-eab8-48a3-828c-6862abe1b648 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:20.212898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:20.217864 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 706/1396] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:17:20.582120 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a8471-fe5c-43fa-bb4a-fc108327b5e9 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:20.583402 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a8471-fe5c-43fa-bb4a-fc108327b5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:20.583573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a8471-fe5c-43fa-bb4a-fc108327b5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:20.693398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-2a1a8471-fe5c-43fa-bb4a-fc108327b5e9 None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:20.695382 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-2a1a8471-fe5c-43fa-bb4a-fc108327b5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:20.839753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:20.848205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 691/1397] 10.222.0.88 () {58 vars in 1009 bytes} [Tue Aug 23 22:17:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:20.904877 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba6a74-518d-450a-9086-989d83c76df8 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:20.905635 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba6a74-518d-450a-9086-989d83c76df8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:20.905635 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba6a74-518d-450a-9086-989d83c76df8 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:20.907161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 707/1398] 10.222.0.88 () {54 vars in 943 bytes} [Tue Aug 23 22:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:17:20.915431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b3357-a93d-4702-ab44-2291f43f699b None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:20.915761 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b3357-a93d-4702-ab44-2291f43f699b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:20.916142 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b3357-a93d-4702-ab44-2291f43f699b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:21.028302 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5b5b3357-a93d-4702-ab44-2291f43f699b None None] MFA Rules not processed for user `601a3defca70428689ee22517d7bcb19`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:21.029913 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5b5b3357-a93d-4702-ab44-2291f43f699b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:21.185171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 692/1399] 10.222.0.88 () {58 vars in 1040 bytes} [Tue Aug 23 22:17:20 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:21.195117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba6a74-518d-450a-9086-989d83c76df8 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:21.196645 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-13ba6a74-518d-450a-9086-989d83c76df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:21.373318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba6a74-518d-450a-9086-989d83c76df8 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:21.499616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:21.531593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f665826e-2e73-408d-b30f-aae6f3f81883 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:21.532219 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665826e-2e73-408d-b30f-aae6f3f81883 service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:21.532634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665826e-2e73-408d-b30f-aae6f3f81883 service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:21.532979 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665826e-2e73-408d-b30f-aae6f3f81883 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:21.534813 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f665826e-2e73-408d-b30f-aae6f3f81883 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:21.660385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f665826e-2e73-408d-b30f-aae6f3f81883 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:21.663751 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f665826e-2e73-408d-b30f-aae6f3f81883 service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:21.674360 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 708/1400] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:17:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 22:17:22.583923 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56802f8d-8741-4bfd-97de-e4f75b49a11b None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:22.584680 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56802f8d-8741-4bfd-97de-e4f75b49a11b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:22.584680 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56802f8d-8741-4bfd-97de-e4f75b49a11b None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:22.589659 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 693/1401] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:17:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:17:22.592570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bd664-4289-4f30-b039-ec9fb034ae48 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:22.593126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bd664-4289-4f30-b039-ec9fb034ae48 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:22.593126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bd664-4289-4f30-b039-ec9fb034ae48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:22.702853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ab9bd664-4289-4f30-b039-ec9fb034ae48 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:22.704371 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ab9bd664-4289-4f30-b039-ec9fb034ae48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:22.831035 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:22.890084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 709/1402] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:23.206453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56802f8d-8741-4bfd-97de-e4f75b49a11b None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:23.208852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-56802f8d-8741-4bfd-97de-e4f75b49a11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:23.286840 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:23.448328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56802f8d-8741-4bfd-97de-e4f75b49a11b None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:23.671827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbd08c-c923-4d07-8baf-da66a0978e4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:23.672782 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbd08c-c923-4d07-8baf-da66a0978e4c service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:23.673096 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbd08c-c923-4d07-8baf-da66a0978e4c service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:23.673380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbd08c-c923-4d07-8baf-da66a0978e4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:23.675502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-eebbd08c-c923-4d07-8baf-da66a0978e4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:23.699021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:23.797550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbd08c-c923-4d07-8baf-da66a0978e4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:23.799392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbd08c-c923-4d07-8baf-da66a0978e4c service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:23.825780 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 694/1403] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:17:26.006549 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45847bc-e7d2-4281-8350-a6b55774a796 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:26.007204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45847bc-e7d2-4281-8350-a6b55774a796 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:26.007204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45847bc-e7d2-4281-8350-a6b55774a796 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:26.009104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 710/1404] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:17:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:17:26.014758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1c85f-1954-4365-aa2e-0fdcebb21c3e None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:26.015369 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1c85f-1954-4365-aa2e-0fdcebb21c3e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:26.015369 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1c85f-1954-4365-aa2e-0fdcebb21c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:26.103898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-35e1c85f-1954-4365-aa2e-0fdcebb21c3e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:26.105261 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-35e1c85f-1954-4365-aa2e-0fdcebb21c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:26.222027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:26.237735 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 695/1405] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:17:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:26.543696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45847bc-e7d2-4281-8350-a6b55774a796 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:26.545304 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e45847bc-e7d2-4281-8350-a6b55774a796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:26.737591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45847bc-e7d2-4281-8350-a6b55774a796 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:26.807883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:26.888274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514682a9-9cae-413d-a15f-e60a033696c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:26.889019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514682a9-9cae-413d-a15f-e60a033696c4 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:26.889325 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514682a9-9cae-413d-a15f-e60a033696c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:26.889607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514682a9-9cae-413d-a15f-e60a033696c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:26.891226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-514682a9-9cae-413d-a15f-e60a033696c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:27.008875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514682a9-9cae-413d-a15f-e60a033696c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:27.010558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514682a9-9cae-413d-a15f-e60a033696c4 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:27.032817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 711/1406] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:17:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:17:29.133665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836c78c-c8c8-4bbc-b071-241ac8d0c90a None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:29.134690 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836c78c-c8c8-4bbc-b071-241ac8d0c90a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:29.134938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836c78c-c8c8-4bbc-b071-241ac8d0c90a None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:29.136474 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 696/1407] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:17:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:17:29.143265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c994f-529a-4f9c-97d9-704e52a277c5 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:29.144269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c994f-529a-4f9c-97d9-704e52a277c5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:29.144554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c994f-529a-4f9c-97d9-704e52a277c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:29.236858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-dc0c994f-529a-4f9c-97d9-704e52a277c5 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:29.238402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-dc0c994f-529a-4f9c-97d9-704e52a277c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:29.258269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:29.378056 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 712/1408] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:29.722662 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836c78c-c8c8-4bbc-b071-241ac8d0c90a None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:29.724794 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e836c78c-c8c8-4bbc-b071-241ac8d0c90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:29.958795 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836c78c-c8c8-4bbc-b071-241ac8d0c90a None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:30.111814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:30.173018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ad354-75b5-416c-81b3-2e423aef3077 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:30.174189 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ad354-75b5-416c-81b3-2e423aef3077 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:30.174512 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ad354-75b5-416c-81b3-2e423aef3077 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:30.174695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ad354-75b5-416c-81b3-2e423aef3077 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:30.177120 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ad3ad354-75b5-416c-81b3-2e423aef3077 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:30.305686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ad354-75b5-416c-81b3-2e423aef3077 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:30.307491 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ad354-75b5-416c-81b3-2e423aef3077 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:30.330085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 697/1409] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:17:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:17:39.375410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=37 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:40.326957 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=65 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:45.674574 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e138b-2644-4cfc-82ce-97318289ce09 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:45.674574 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e138b-2644-4cfc-82ce-97318289ce09 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:45.674574 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e138b-2644-4cfc-82ce-97318289ce09 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:45.677075 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 713/1410] 10.222.0.88 () {54 vars in 918 bytes} [Tue Aug 23 22:17:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:17:45.683740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12213ed5-be2f-4e77-a97a-c66d8b54c444 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:45.684608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12213ed5-be2f-4e77-a97a-c66d8b54c444 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:45.684608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12213ed5-be2f-4e77-a97a-c66d8b54c444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:45.780140 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-12213ed5-be2f-4e77-a97a-c66d8b54c444 None None] MFA Rules not processed for user `e642dd7a07ca4afaa40c2769299d8bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:45.782257 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-12213ed5-be2f-4e77-a97a-c66d8b54c444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:45.933470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 698/1411] 10.222.0.88 () {58 vars in 1015 bytes} [Tue Aug 23 22:17:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:45.950816 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a981dd9-2cf4-4a7b-8d93-2d37f6ac55f6 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:45.951087 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a981dd9-2cf4-4a7b-8d93-2d37f6ac55f6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:45.951377 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a981dd9-2cf4-4a7b-8d93-2d37f6ac55f6 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:45.952747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 714/1412] 10.222.0.88 () {54 vars in 918 bytes} [Tue Aug 23 22:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:17:45.960335 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131175dd-2299-4b4b-b814-d045730f5813 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:45.960642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131175dd-2299-4b4b-b814-d045730f5813 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:45.960902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131175dd-2299-4b4b-b814-d045730f5813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:46.053430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:46.057540 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-131175dd-2299-4b4b-b814-d045730f5813 None None] MFA Rules not processed for user `e3517b3b077a4a1fa76163657b46930c`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:46.059739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-131175dd-2299-4b4b-b814-d045730f5813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:46.200297 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 699/1413] 10.222.0.88 () {58 vars in 1015 bytes} [Tue Aug 23 22:17:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:48.805725 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c82969-2ed8-44a1-ad8d-bce32829ab9f None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:48.806551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c82969-2ed8-44a1-ad8d-bce32829ab9f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:48.806745 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c82969-2ed8-44a1-ad8d-bce32829ab9f None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:48.808316 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:17:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:17:48.814626 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcdc5e-1740-4422-9088-efddd1ae79f6 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:48.815120 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcdc5e-1740-4422-9088-efddd1ae79f6 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:48.815226 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcdc5e-1740-4422-9088-efddd1ae79f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:48.889334 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-0bbcdc5e-1740-4422-9088-efddd1ae79f6 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:48.890583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0bbcdc5e-1740-4422-9088-efddd1ae79f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:48.953643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:49.012046 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:49.315796 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c82969-2ed8-44a1-ad8d-bce32829ab9f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:49.318499 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-31c82969-2ed8-44a1-ad8d-bce32829ab9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:49.511893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c82969-2ed8-44a1-ad8d-bce32829ab9f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:49.657197 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:49.708842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab4cfd-bca0-48f0-bf93-125647a5610e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:49.709699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab4cfd-bca0-48f0-bf93-125647a5610e service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:49.710172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab4cfd-bca0-48f0-bf93-125647a5610e service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:49.710607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab4cfd-bca0-48f0-bf93-125647a5610e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:49.712674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3aab4cfd-bca0-48f0-bf93-125647a5610e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:49.842037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab4cfd-bca0-48f0-bf93-125647a5610e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:49.844386 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab4cfd-bca0-48f0-bf93-125647a5610e service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:49.870888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 716/1416] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:17:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:17:50.945844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a788819-76e8-4350-a156-1e95a8c908aa None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:50.946726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a788819-76e8-4350-a156-1e95a8c908aa None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:50.946902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a788819-76e8-4350-a156-1e95a8c908aa None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:50.948454 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:17:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:17:50.956019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedcfe2-2be7-4278-9103-174f2fb5b841 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:50.956642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedcfe2-2be7-4278-9103-174f2fb5b841 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:50.956999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedcfe2-2be7-4278-9103-174f2fb5b841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:51.056337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-adedcfe2-2be7-4278-9103-174f2fb5b841 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:51.058433 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-adedcfe2-2be7-4278-9103-174f2fb5b841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:51.104348 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:51.203469 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:51.359051 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a788819-76e8-4350-a156-1e95a8c908aa None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:51.360488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3a788819-76e8-4350-a156-1e95a8c908aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:51.572751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a788819-76e8-4350-a156-1e95a8c908aa None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:51.632739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:51.741561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ebef19-86ed-4503-9167-e5c35dad0b79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:51.742201 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebef19-86ed-4503-9167-e5c35dad0b79 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:51.742485 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebef19-86ed-4503-9167-e5c35dad0b79 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:51.742752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebef19-86ed-4503-9167-e5c35dad0b79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:51.744458 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-98ebef19-86ed-4503-9167-e5c35dad0b79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:51.877019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ebef19-86ed-4503-9167-e5c35dad0b79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:51.878765 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ebef19-86ed-4503-9167-e5c35dad0b79 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:51.909506 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 702/1419] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:17:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:17:52.961779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749669e-bc3b-427e-bfd9-ac486341535f None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:52.962467 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749669e-bc3b-427e-bfd9-ac486341535f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:52.962467 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749669e-bc3b-427e-bfd9-ac486341535f None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:52.963894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:17:52.970345 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c2fd5-ca02-42b4-83b6-888f361db8f2 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:52.971156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c2fd5-ca02-42b4-83b6-888f361db8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:52.971156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c2fd5-ca02-42b4-83b6-888f361db8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:53.051089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:53.085201 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-8f8c2fd5-ca02-42b4-83b6-888f361db8f2 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:53.086588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8f8c2fd5-ca02-42b4-83b6-888f361db8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:53.243190 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:53.403985 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749669e-bc3b-427e-bfd9-ac486341535f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:53.407876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7749669e-bc3b-427e-bfd9-ac486341535f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:53.603803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749669e-bc3b-427e-bfd9-ac486341535f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:53.619345 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:53.750690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe9562-d65c-4317-8345-83ad8916cc0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:53.751634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe9562-d65c-4317-8345-83ad8916cc0f service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:53.752148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe9562-d65c-4317-8345-83ad8916cc0f service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:53.752651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe9562-d65c-4317-8345-83ad8916cc0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:53.756476 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-71fe9562-d65c-4317-8345-83ad8916cc0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:53.887600 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe9562-d65c-4317-8345-83ad8916cc0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:53.889568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe9562-d65c-4317-8345-83ad8916cc0f service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:53.911913 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 719/1422] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:17:54.860257 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cf59f-6297-47b4-b989-a9c74b2b2ad9 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:54.860857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cf59f-6297-47b4-b989-a9c74b2b2ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:54.860985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cf59f-6297-47b4-b989-a9c74b2b2ad9 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:54.862699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:17:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:17:54.869718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29409ef1-d7df-42c3-b873-e4253e8e1e05 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:54.870316 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29409ef1-d7df-42c3-b873-e4253e8e1e05 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:54.870517 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29409ef1-d7df-42c3-b873-e4253e8e1e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:54.899845 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:54.967613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-29409ef1-d7df-42c3-b873-e4253e8e1e05 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:54.968863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-29409ef1-d7df-42c3-b873-e4253e8e1e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:55.121442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:17:55.297706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cf59f-6297-47b4-b989-a9c74b2b2ad9 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:55.299744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3b8cf59f-6297-47b4-b989-a9c74b2b2ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:55.487937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:55.551818 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cf59f-6297-47b4-b989-a9c74b2b2ad9 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:55.732512 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd88ada-2a20-49a5-84d7-c4ed4953ce98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:55.733191 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd88ada-2a20-49a5-84d7-c4ed4953ce98 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:55.733443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd88ada-2a20-49a5-84d7-c4ed4953ce98 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:55.733949 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd88ada-2a20-49a5-84d7-c4ed4953ce98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:55.735608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-fdd88ada-2a20-49a5-84d7-c4ed4953ce98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:55.831342 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:55.846112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd88ada-2a20-49a5-84d7-c4ed4953ce98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:55.847738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd88ada-2a20-49a5-84d7-c4ed4953ce98 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:55.868563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 705/1425] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:17:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:17:56.861568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d86b5-de83-4625-8afe-ae74bd40e375 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:56.862188 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d86b5-de83-4625-8afe-ae74bd40e375 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:56.862188 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d86b5-de83-4625-8afe-ae74bd40e375 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:56.863491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:17:56.873630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690e285-da1a-41b9-a055-716933177ba9 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:56.873630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690e285-da1a-41b9-a055-716933177ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:56.873630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690e285-da1a-41b9-a055-716933177ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:56.968808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-0690e285-da1a-41b9-a055-716933177ba9 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:56.970089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0690e285-da1a-41b9-a055-716933177ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:57.094498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:57.296680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d86b5-de83-4625-8afe-ae74bd40e375 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:57.298411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-010d86b5-de83-4625-8afe-ae74bd40e375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:57.426498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:57.518107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d86b5-de83-4625-8afe-ae74bd40e375 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:57.675396 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beefccae-cef7-4a6f-9827-c89f5b641b4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:57.676026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefccae-cef7-4a6f-9827-c89f5b641b4c service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:57.676481 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefccae-cef7-4a6f-9827-c89f5b641b4c service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:57.676909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefccae-cef7-4a6f-9827-c89f5b641b4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:57.678743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-beefccae-cef7-4a6f-9827-c89f5b641b4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:57.744563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:57.803910 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beefccae-cef7-4a6f-9827-c89f5b641b4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:57.805605 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beefccae-cef7-4a6f-9827-c89f5b641b4c service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:57.829225 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 722/1428] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:17:58.777220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5292474-fcf3-4179-a898-f3f1a55a899c None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:58.778147 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5292474-fcf3-4179-a898-f3f1a55a899c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:58.778147 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5292474-fcf3-4179-a898-f3f1a55a899c None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:58.780176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:17:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:17:58.786825 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef42ef-f6fc-4f38-955c-50685e1fe2d6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:58.787410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef42ef-f6fc-4f38-955c-50685e1fe2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:58.787410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef42ef-f6fc-4f38-955c-50685e1fe2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:58.882285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7cef42ef-f6fc-4f38-955c-50685e1fe2d6 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:17:58.884440 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7cef42ef-f6fc-4f38-955c-50685e1fe2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:59.038188 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:17:59.196457 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5292474-fcf3-4179-a898-f3f1a55a899c None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:17:59.198084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d5292474-fcf3-4179-a898-f3f1a55a899c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:59.263883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:59.391690 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5292474-fcf3-4179-a898-f3f1a55a899c None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:17:59.540810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5ed10-451c-4850-84c1-e7f31a730db0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:17:59.541688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5ed10-451c-4850-84c1-e7f31a730db0 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:17:59.542167 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5ed10-451c-4850-84c1-e7f31a730db0 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:17:59.542492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5ed10-451c-4850-84c1-e7f31a730db0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:17:59.544467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a8f5ed10-451c-4850-84c1-e7f31a730db0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:17:59.553253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:17:59.657912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5ed10-451c-4850-84c1-e7f31a730db0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:17:59.659533 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5ed10-451c-4850-84c1-e7f31a730db0 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:17:59.680569 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 708/1431] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:17:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:18:00.506771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c8366-8589-417a-b17c-543468e13be2 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:00.507449 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c8366-8589-417a-b17c-543468e13be2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:00.507761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c8366-8589-417a-b17c-543468e13be2 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:00.510460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:18:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:18:00.516427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877c63e-f574-4f03-8d43-23152821fc62 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:00.516925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877c63e-f574-4f03-8d43-23152821fc62 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:00.517021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877c63e-f574-4f03-8d43-23152821fc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:00.604460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5877c63e-f574-4f03-8d43-23152821fc62 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:18:00.606080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5877c63e-f574-4f03-8d43-23152821fc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:00.681065 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:00.712405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:18:00.866013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c8366-8589-417a-b17c-543468e13be2 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:18:00.867596 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fd6c8366-8589-417a-b17c-543468e13be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:00.886199 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:01.023711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c8366-8589-417a-b17c-543468e13be2 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:18:01.123469 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:01.153995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49313550-13d3-40ef-a249-8d79c552fe32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:18:01.154623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49313550-13d3-40ef-a249-8d79c552fe32 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:01.154900 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49313550-13d3-40ef-a249-8d79c552fe32 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:01.155171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49313550-13d3-40ef-a249-8d79c552fe32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:01.156857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-49313550-13d3-40ef-a249-8d79c552fe32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:01.251310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49313550-13d3-40ef-a249-8d79c552fe32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:18:01.252953 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49313550-13d3-40ef-a249-8d79c552fe32 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:18:01.271841 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 725/1434] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:18:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:18:02.098637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5122da9-3504-4e8e-8452-cd344d8a722e None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:02.099159 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5122da9-3504-4e8e-8452-cd344d8a722e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:02.099413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5122da9-3504-4e8e-8452-cd344d8a722e None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:02.100794 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:18:02.106100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64e84a-0481-4be9-b43e-7c316887b504 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:02.106592 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64e84a-0481-4be9-b43e-7c316887b504 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:02.106698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64e84a-0481-4be9-b43e-7c316887b504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:02.186843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-cc64e84a-0481-4be9-b43e-7c316887b504 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:18:02.188203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-cc64e84a-0481-4be9-b43e-7c316887b504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:02.233587 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:02.296690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:18:02.451500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5122da9-3504-4e8e-8452-cd344d8a722e None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:18:02.453033 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c5122da9-3504-4e8e-8452-cd344d8a722e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:02.651452 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5122da9-3504-4e8e-8452-cd344d8a722e None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:18:02.714141 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:02.800368 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a656279-a780-4b77-91c8-46ed0bec2a0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:18:02.801305 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a656279-a780-4b77-91c8-46ed0bec2a0d service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:02.801802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a656279-a780-4b77-91c8-46ed0bec2a0d service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:02.802266 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a656279-a780-4b77-91c8-46ed0bec2a0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:02.804382 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8a656279-a780-4b77-91c8-46ed0bec2a0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:02.921313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a656279-a780-4b77-91c8-46ed0bec2a0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:18:02.922969 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a656279-a780-4b77-91c8-46ed0bec2a0d service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:18:02.944360 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 711/1437] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:18:03.822606 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e202f67-c1da-4bb4-9def-680c970cfda5 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:03.823205 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e202f67-c1da-4bb4-9def-680c970cfda5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:03.823205 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e202f67-c1da-4bb4-9def-680c970cfda5 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:03.824575 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:18:03.830488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d0981-fbc5-4834-8c8b-1998e1b9932c None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:03.831315 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d0981-fbc5-4834-8c8b-1998e1b9932c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:03.831513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d0981-fbc5-4834-8c8b-1998e1b9932c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:03.917919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-a10d0981-fbc5-4834-8c8b-1998e1b9932c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:18:03.919195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a10d0981-fbc5-4834-8c8b-1998e1b9932c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:03.921703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:04.030519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:18:04.182226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e202f67-c1da-4bb4-9def-680c970cfda5 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:18:04.183979 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4e202f67-c1da-4bb4-9def-680c970cfda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:04.343996 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e202f67-c1da-4bb4-9def-680c970cfda5 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:18:04.372487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:04.465770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a4228-6659-470f-83c8-52b179e882cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:18:04.466607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a4228-6659-470f-83c8-52b179e882cb service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:04.466882 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a4228-6659-470f-83c8-52b179e882cb service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:04.467159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a4228-6659-470f-83c8-52b179e882cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:04.469375 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9f6a4228-6659-470f-83c8-52b179e882cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:04.571967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a4228-6659-470f-83c8-52b179e882cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:18:04.573664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a4228-6659-470f-83c8-52b179e882cb service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:18:04.593758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 728/1440] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:18:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:18:06.459706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d15e66-ad5a-487d-a792-0a9f109e96fd None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:06.460316 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d15e66-ad5a-487d-a792-0a9f109e96fd None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:06.460316 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d15e66-ad5a-487d-a792-0a9f109e96fd None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:06.461634 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:18:06.467922 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288de53-2960-4be1-adcd-1c478dea4e69 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:06.468478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288de53-2960-4be1-adcd-1c478dea4e69 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:06.468603 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288de53-2960-4be1-adcd-1c478dea4e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:06.521391 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:06.557184 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0288de53-2960-4be1-adcd-1c478dea4e69 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:18:06.558490 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0288de53-2960-4be1-adcd-1c478dea4e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:06.664197 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:18:06.955142 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d15e66-ad5a-487d-a792-0a9f109e96fd None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:18:06.956701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-06d15e66-ad5a-487d-a792-0a9f109e96fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:07.066056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d15e66-ad5a-487d-a792-0a9f109e96fd None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:18:07.103922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:07.150260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81c57f-0ce5-49db-89f5-2b193dd3e94b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:18:07.150873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81c57f-0ce5-49db-89f5-2b193dd3e94b service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:07.151151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81c57f-0ce5-49db-89f5-2b193dd3e94b service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:07.151417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81c57f-0ce5-49db-89f5-2b193dd3e94b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:07.153030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-2d81c57f-0ce5-49db-89f5-2b193dd3e94b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:07.272430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81c57f-0ce5-49db-89f5-2b193dd3e94b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:18:07.273966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81c57f-0ce5-49db-89f5-2b193dd3e94b service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:18:07.416417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 714/1443] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:18:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:18:09.746117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347efbe-b77a-4910-9b2e-d55efffe4f12 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:09.746725 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347efbe-b77a-4910-9b2e-d55efffe4f12 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:09.746725 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347efbe-b77a-4910-9b2e-d55efffe4f12 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:09.748525 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:18:09.754193 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def699c-22be-40c0-b648-4038ac819abb None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:09.755393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def699c-22be-40c0-b648-4038ac819abb None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:09.755393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def699c-22be-40c0-b648-4038ac819abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:09.833768 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-6def699c-22be-40c0-b648-4038ac819abb None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:18:09.835005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6def699c-22be-40c0-b648-4038ac819abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:09.849715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:09.944058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:18:10.243699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347efbe-b77a-4910-9b2e-d55efffe4f12 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:18:10.245353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a347efbe-b77a-4910-9b2e-d55efffe4f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:10.367521 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347efbe-b77a-4910-9b2e-d55efffe4f12 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:18:10.380631 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:10.455003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eccebc-f1c9-4277-ba8f-85bc019cd358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:18:10.455629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eccebc-f1c9-4277-ba8f-85bc019cd358 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:10.456081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eccebc-f1c9-4277-ba8f-85bc019cd358 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:10.456358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eccebc-f1c9-4277-ba8f-85bc019cd358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:10.458582 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-86eccebc-f1c9-4277-ba8f-85bc019cd358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:10.630901 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eccebc-f1c9-4277-ba8f-85bc019cd358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:18:10.632903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eccebc-f1c9-4277-ba8f-85bc019cd358 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:18:10.655808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 731/1446] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:18:13.033430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3d356-989a-46c1-bbc0-5c413e5a74b5 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:13.034144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3d356-989a-46c1-bbc0-5c413e5a74b5 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:13.034144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3d356-989a-46c1-bbc0-5c413e5a74b5 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:13.035934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:18:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:18:13.042751 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb447fc0-85bd-4e62-ad51-146afc458513 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:13.043266 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb447fc0-85bd-4e62-ad51-146afc458513 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:13.043358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb447fc0-85bd-4e62-ad51-146afc458513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:13.102930 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:13.123269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-cb447fc0-85bd-4e62-ad51-146afc458513 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:18:13.124508 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-cb447fc0-85bd-4e62-ad51-146afc458513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:13.235513 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:18:13.546331 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3d356-989a-46c1-bbc0-5c413e5a74b5 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:18:13.547944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-1ad3d356-989a-46c1-bbc0-5c413e5a74b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:13.668880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3d356-989a-46c1-bbc0-5c413e5a74b5 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:18:13.714613 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:13.760737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c9bd6-07c9-43c1-a737-00090f098ad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:18:13.761404 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c9bd6-07c9-43c1-a737-00090f098ad7 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:13.765370 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c9bd6-07c9-43c1-a737-00090f098ad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:13.765370 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c9bd6-07c9-43c1-a737-00090f098ad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:13.765370 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-473c9bd6-07c9-43c1-a737-00090f098ad7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:13.900371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c9bd6-07c9-43c1-a737-00090f098ad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:18:13.901847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c9bd6-07c9-43c1-a737-00090f098ad7 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:18:13.921362 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 717/1449] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:18:16.318442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827a871-28f9-49be-be80-128bf91f5632 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:16.319305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827a871-28f9-49be-be80-128bf91f5632 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:16.319305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827a871-28f9-49be-be80-128bf91f5632 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:16.320405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:18:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:18:16.326414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36876b-5f22-4d22-bb65-4be8f1944469 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:16.326910 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36876b-5f22-4d22-bb65-4be8f1944469 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:16.327017 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36876b-5f22-4d22-bb65-4be8f1944469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:16.401346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-8c36876b-5f22-4d22-bb65-4be8f1944469 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:18:16.402642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8c36876b-5f22-4d22-bb65-4be8f1944469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:16.425802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:16.510380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:18:16.808067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827a871-28f9-49be-be80-128bf91f5632 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:18:16.810373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2827a871-28f9-49be-be80-128bf91f5632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:16.923187 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827a871-28f9-49be-be80-128bf91f5632 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:18:16.946725 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:17.010573 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e39fe-27ae-4d0c-9b21-9a05d9c922b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:18:17.011185 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e39fe-27ae-4d0c-9b21-9a05d9c922b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:17.011472 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e39fe-27ae-4d0c-9b21-9a05d9c922b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:17.011742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e39fe-27ae-4d0c-9b21-9a05d9c922b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:17.013430 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6b5e39fe-27ae-4d0c-9b21-9a05d9c922b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:17.155545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e39fe-27ae-4d0c-9b21-9a05d9c922b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:18:17.157087 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e39fe-27ae-4d0c-9b21-9a05d9c922b8 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:18:17.188127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 734/1452] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:18:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:18:19.295374 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a381618-b48d-4192-92c7-5e06d814a625 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:19.295958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a381618-b48d-4192-92c7-5e06d814a625 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:19.296084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a381618-b48d-4192-92c7-5e06d814a625 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:19.297398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:18:19.303086 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3582cf9-4677-4b1f-9b85-4ebf3836bbfe None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:19.303613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3582cf9-4677-4b1f-9b85-4ebf3836bbfe None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:19.303734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3582cf9-4677-4b1f-9b85-4ebf3836bbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:19.372165 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:19.375735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f3582cf9-4677-4b1f-9b85-4ebf3836bbfe None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:18:19.377453 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f3582cf9-4677-4b1f-9b85-4ebf3836bbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:19.484441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:18:19.782092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a381618-b48d-4192-92c7-5e06d814a625 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:18:19.783631 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5a381618-b48d-4192-92c7-5e06d814a625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:19.894537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a381618-b48d-4192-92c7-5e06d814a625 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:18:19.947553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:19.973832 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ab745-6cbe-4ae3-8110-29814c3ebce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:18:19.974452 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ab745-6cbe-4ae3-8110-29814c3ebce2 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:19.974732 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ab745-6cbe-4ae3-8110-29814c3ebce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:19.975001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ab745-6cbe-4ae3-8110-29814c3ebce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:19.976636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-357ab745-6cbe-4ae3-8110-29814c3ebce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:20.089947 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ab745-6cbe-4ae3-8110-29814c3ebce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:18:20.091308 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ab745-6cbe-4ae3-8110-29814c3ebce2 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:18:20.111284 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 720/1455] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:18:22.138372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192865f-2fd4-4a44-9466-306485dc628d None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:22.139076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192865f-2fd4-4a44-9466-306485dc628d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:22.139076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192865f-2fd4-4a44-9466-306485dc628d None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:22.140549 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:18:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:18:22.146824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de9c1f-b082-4945-b51c-707bc63f1de3 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:22.147424 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de9c1f-b082-4945-b51c-707bc63f1de3 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:22.147424 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de9c1f-b082-4945-b51c-707bc63f1de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:22.228839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-89de9c1f-b082-4945-b51c-707bc63f1de3 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:18:22.230111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-89de9c1f-b082-4945-b51c-707bc63f1de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:22.265796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:22.345393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:18:22.639944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192865f-2fd4-4a44-9466-306485dc628d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:18:22.641552 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6192865f-2fd4-4a44-9466-306485dc628d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:22.763378 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192865f-2fd4-4a44-9466-306485dc628d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:18:22.796281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:22.858204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231d55f-1720-4748-bbd6-b6657b3cd602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:18:22.858819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231d55f-1720-4748-bbd6-b6657b3cd602 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:22.859106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231d55f-1720-4748-bbd6-b6657b3cd602 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:22.859376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231d55f-1720-4748-bbd6-b6657b3cd602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:22.861024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e231d55f-1720-4748-bbd6-b6657b3cd602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:22.990729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231d55f-1720-4748-bbd6-b6657b3cd602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:18:22.992196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231d55f-1720-4748-bbd6-b6657b3cd602 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:18:23.013987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 737/1458] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:18:25.127881 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8238db5-e7c0-4c9c-81c6-26dbf2a61a84 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:25.128435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8238db5-e7c0-4c9c-81c6-26dbf2a61a84 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:25.128692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8238db5-e7c0-4c9c-81c6-26dbf2a61a84 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:25.131176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:18:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:18:25.138184 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c26c1b-1f4a-400a-9d68-cfffbac4d4bc None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:25.138995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c26c1b-1f4a-400a-9d68-cfffbac4d4bc None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:25.139156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c26c1b-1f4a-400a-9d68-cfffbac4d4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:25.368980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b7c26c1b-1f4a-400a-9d68-cfffbac4d4bc None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:18:25.371182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b7c26c1b-1f4a-400a-9d68-cfffbac4d4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:25.389704 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:25.571936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:18:26.140118 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8238db5-e7c0-4c9c-81c6-26dbf2a61a84 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:18:26.141802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c8238db5-e7c0-4c9c-81c6-26dbf2a61a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:26.271364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8238db5-e7c0-4c9c-81c6-26dbf2a61a84 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:18:26.351365 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:26.375678 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6cac9-a23e-4507-8a25-35f239a44d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:18:26.376460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6cac9-a23e-4507-8a25-35f239a44d61 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:26.376797 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6cac9-a23e-4507-8a25-35f239a44d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:26.377267 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6cac9-a23e-4507-8a25-35f239a44d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:26.380628 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-91c6cac9-a23e-4507-8a25-35f239a44d61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:26.544321 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6cac9-a23e-4507-8a25-35f239a44d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:18:26.546061 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6cac9-a23e-4507-8a25-35f239a44d61 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:18:26.573280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 723/1461] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:18:28.854522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f495e-defa-43f6-a809-4234be373e79 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:28.855388 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f495e-defa-43f6-a809-4234be373e79 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:28.855388 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f495e-defa-43f6-a809-4234be373e79 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:28.857185 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:18:28.863416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de7f63-5f9a-416f-bf11-d27346923b01 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:28.864185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de7f63-5f9a-416f-bf11-d27346923b01 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:28.864185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de7f63-5f9a-416f-bf11-d27346923b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:28.969871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-63de7f63-5f9a-416f-bf11-d27346923b01 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:18:28.971498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-63de7f63-5f9a-416f-bf11-d27346923b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:29.031847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:29.136968 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:18:29.465143 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f495e-defa-43f6-a809-4234be373e79 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:18:29.466677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f56f495e-defa-43f6-a809-4234be373e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:29.580654 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f495e-defa-43f6-a809-4234be373e79 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:18:29.623253 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:29.684770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3a886-99e1-4426-b764-ad2fe447d674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:18:29.685885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3a886-99e1-4426-b764-ad2fe447d674 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:29.686441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3a886-99e1-4426-b764-ad2fe447d674 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:29.686855 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3a886-99e1-4426-b764-ad2fe447d674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:29.689590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b3a3a886-99e1-4426-b764-ad2fe447d674 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:29.824540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3a886-99e1-4426-b764-ad2fe447d674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:18:29.826022 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3a886-99e1-4426-b764-ad2fe447d674 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:18:29.845980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 740/1464] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:18:31.904382 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d1e66-930a-447a-bcf6-76cb044a00ff None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:31.904920 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d1e66-930a-447a-bcf6-76cb044a00ff None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:31.904920 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d1e66-930a-447a-bcf6-76cb044a00ff None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:31.906311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:18:31.912012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357db3aa-dd84-4f7b-91bc-83c772c33794 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:31.912752 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357db3aa-dd84-4f7b-91bc-83c772c33794 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:31.912752 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357db3aa-dd84-4f7b-91bc-83c772c33794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:31.994914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-357db3aa-dd84-4f7b-91bc-83c772c33794 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:18:31.996148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-357db3aa-dd84-4f7b-91bc-83c772c33794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:32.018331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:32.115173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:18:32.413036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d1e66-930a-447a-bcf6-76cb044a00ff None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:18:32.414644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-b78d1e66-930a-447a-bcf6-76cb044a00ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:32.523618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d1e66-930a-447a-bcf6-76cb044a00ff None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:18:32.599819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:32.608696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c5c6d-4d2c-4503-afd8-9b068d23effd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:18:32.609348 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c5c6d-4d2c-4503-afd8-9b068d23effd service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:32.609626 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c5c6d-4d2c-4503-afd8-9b068d23effd service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:32.609894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c5c6d-4d2c-4503-afd8-9b068d23effd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:32.611597 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-122c5c6d-4d2c-4503-afd8-9b068d23effd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:32.732538 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c5c6d-4d2c-4503-afd8-9b068d23effd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:18:32.733933 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c5c6d-4d2c-4503-afd8-9b068d23effd service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:18:32.754015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 726/1467] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:18:34.731118 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a0cda-c0fd-43eb-b731-b4fc930465ac None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:34.731698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a0cda-c0fd-43eb-b731-b4fc930465ac None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:34.731698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a0cda-c0fd-43eb-b731-b4fc930465ac None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:34.733032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:18:34.739510 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0698eb5-2a12-4e6a-8a03-e29223081b1a None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:34.740058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0698eb5-2a12-4e6a-8a03-e29223081b1a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:34.740058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0698eb5-2a12-4e6a-8a03-e29223081b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:34.811735 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-e0698eb5-2a12-4e6a-8a03-e29223081b1a None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:18:34.812963 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e0698eb5-2a12-4e6a-8a03-e29223081b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:34.862316 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:34.913225 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:18:35.200980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a0cda-c0fd-43eb-b731-b4fc930465ac None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:18:35.202594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e68a0cda-c0fd-43eb-b731-b4fc930465ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:35.313570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a0cda-c0fd-43eb-b731-b4fc930465ac None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:18:35.362296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:35.390677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fdb6f-5768-4e89-86d9-9acc4e279055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:18:35.391287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fdb6f-5768-4e89-86d9-9acc4e279055 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:35.391563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fdb6f-5768-4e89-86d9-9acc4e279055 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:35.391830 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fdb6f-5768-4e89-86d9-9acc4e279055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:35.393466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-192fdb6f-5768-4e89-86d9-9acc4e279055 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:35.506456 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fdb6f-5768-4e89-86d9-9acc4e279055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:18:35.507901 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fdb6f-5768-4e89-86d9-9acc4e279055 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:18:35.527679 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 743/1470] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:18:44.911164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=22 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:45.525265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=66 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:57.611253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e14ea-e341-4e75-a6b0-bdafd1663bb8 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:57.611922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e14ea-e341-4e75-a6b0-bdafd1663bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:57.611922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e14ea-e341-4e75-a6b0-bdafd1663bb8 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:57.613877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 728/1471] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:18:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:18:57.621023 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9433ea3-9585-4b11-8a85-342fe10fa411 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:57.621641 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9433ea3-9585-4b11-8a85-342fe10fa411 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:57.621789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9433ea3-9585-4b11-8a85-342fe10fa411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:57.694917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f9433ea3-9585-4b11-8a85-342fe10fa411 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:18:57.696150 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f9433ea3-9585-4b11-8a85-342fe10fa411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:57.813274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 744/1472] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:18:58.148111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d01975-1051-4fbb-a450-1d0d31248510 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:58.148711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d01975-1051-4fbb-a450-1d0d31248510 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:58.149132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d01975-1051-4fbb-a450-1d0d31248510 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:58.151032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 729/1473] 10.222.0.88 () {54 vars in 967 bytes} [Tue Aug 23 22:18:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:18:58.157371 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55d5ec-8743-47e5-a2c4-75fc2649afed None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:58.157658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55d5ec-8743-47e5-a2c4-75fc2649afed None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:58.157933 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55d5ec-8743-47e5-a2c4-75fc2649afed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:58.242927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-5e55d5ec-8743-47e5-a2c4-75fc2649afed None None] MFA Rules not processed for user `2211e237fe554a82baf129e440a0fb43`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:18:58.244172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5e55d5ec-8743-47e5-a2c4-75fc2649afed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:58.265545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:58.329471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 745/1474] 10.222.0.88 () {58 vars in 1064 bytes} [Tue Aug 23 22:18:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:18:58.337793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d01975-1051-4fbb-a450-1d0d31248510 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:18:58.339382 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-63d01975-1051-4fbb-a450-1d0d31248510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:58.457748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d01975-1051-4fbb-a450-1d0d31248510 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:18:58.554832 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc353b-9938-446a-8bb4-59abe0883365 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2211e237fe554a82baf129e440a0fb43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:18:58.555476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc353b-9938-446a-8bb4-59abe0883365 service glance] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:18:58.555940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc353b-9938-446a-8bb4-59abe0883365 service glance] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:18:58.556221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc353b-9938-446a-8bb4-59abe0883365 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:18:58.557928 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-aafc353b-9938-446a-8bb4-59abe0883365 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:18:58.631855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:18:58.702265 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc353b-9938-446a-8bb4-59abe0883365 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:18:58.703908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc353b-9938-446a-8bb4-59abe0883365 service glance] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:18:58.728762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 730/1475] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:19:00.850371 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0caac1-422e-4334-b891-aa2c81f45a50 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:00.851080 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0caac1-422e-4334-b891-aa2c81f45a50 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:00.851080 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0caac1-422e-4334-b891-aa2c81f45a50 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:00.852282 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 746/1476] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:19:00.858324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef4fe9-8cc7-48ae-ae12-01be5b04d112 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:00.858901 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef4fe9-8cc7-48ae-ae12-01be5b04d112 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:00.858901 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef4fe9-8cc7-48ae-ae12-01be5b04d112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:00.931679 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-68ef4fe9-8cc7-48ae-ae12-01be5b04d112 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:19:00.932919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-68ef4fe9-8cc7-48ae-ae12-01be5b04d112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:01.035015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 731/1477] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:19:01.310721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0caac1-422e-4334-b891-aa2c81f45a50 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:19:01.312260 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fb0caac1-422e-4334-b891-aa2c81f45a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:01.450617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0caac1-422e-4334-b891-aa2c81f45a50 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:19:01.503603 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:19:01.555022 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb9d23-cb8f-4c1b-b637-8597cbb18f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:19:01.555650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb9d23-cb8f-4c1b-b637-8597cbb18f1b admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:01.555922 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb9d23-cb8f-4c1b-b637-8597cbb18f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:01.556197 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb9d23-cb8f-4c1b-b637-8597cbb18f1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:01.556848 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb9d23-cb8f-4c1b-b637-8597cbb18f1b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:19:01.558688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: WARNING py.warnings [None req-23fb9d23-cb8f-4c1b-b637-8597cbb18f1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:19:01.558688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: warnings.warn(msg) Aug 23 22:19:01.558688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: Aug 23 22:19:01.560113 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb9d23-cb8f-4c1b-b637-8597cbb18f1b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:19:01.582424 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 747/1478] 10.222.0.88 () {56 vars in 1108 bytes} [Tue Aug 23 22:19:01 2022] GET /identity/v3/projects => generated 2553 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:19:03.583882 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1eb03-4d7a-4074-90e5-eae9bb507a9e None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:03.584609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1eb03-4d7a-4074-90e5-eae9bb507a9e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:03.584609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1eb03-4d7a-4074-90e5-eae9bb507a9e None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:03.585975 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 732/1479] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:19:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:19:03.592548 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c36ce-fd3f-4bf6-8230-4ebe2f1cfcc6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:03.593566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c36ce-fd3f-4bf6-8230-4ebe2f1cfcc6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:03.593891 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c36ce-fd3f-4bf6-8230-4ebe2f1cfcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:03.670476 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-135c36ce-fd3f-4bf6-8230-4ebe2f1cfcc6 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:19:03.671720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-135c36ce-fd3f-4bf6-8230-4ebe2f1cfcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:03.790173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 748/1480] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:19:04.113727 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1eb03-4d7a-4074-90e5-eae9bb507a9e None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:19:04.116176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0ea1eb03-4d7a-4074-90e5-eae9bb507a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:04.126985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:19:04.276491 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1eb03-4d7a-4074-90e5-eae9bb507a9e None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:19:04.368037 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:19:04.410175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb47570-2b15-4a86-bb64-4891e21ddc6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:19:04.410794 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb47570-2b15-4a86-bb64-4891e21ddc6f service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:04.411082 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb47570-2b15-4a86-bb64-4891e21ddc6f service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:04.411359 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb47570-2b15-4a86-bb64-4891e21ddc6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:04.413024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-dcb47570-2b15-4a86-bb64-4891e21ddc6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:04.508143 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb47570-2b15-4a86-bb64-4891e21ddc6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:19:04.509802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb47570-2b15-4a86-bb64-4891e21ddc6f service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:19:04.528749 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 733/1481] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:19:06.716893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cefaac-bacd-441d-b53a-7301c97c62f5 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:06.717480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cefaac-bacd-441d-b53a-7301c97c62f5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:06.717480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cefaac-bacd-441d-b53a-7301c97c62f5 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:06.719284 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 749/1482] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:19:06.728710 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd1e9f-65c0-4077-b8b3-7b574b9e06e5 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:06.729590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd1e9f-65c0-4077-b8b3-7b574b9e06e5 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:06.729590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd1e9f-65c0-4077-b8b3-7b574b9e06e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:06.821325 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-abfd1e9f-65c0-4077-b8b3-7b574b9e06e5 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:19:06.822703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-abfd1e9f-65c0-4077-b8b3-7b574b9e06e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:06.888224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:19:06.953524 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 734/1483] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:19:07.258268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cefaac-bacd-441d-b53a-7301c97c62f5 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:19:07.259851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c1cefaac-bacd-441d-b53a-7301c97c62f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:07.288492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:19:07.382613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cefaac-bacd-441d-b53a-7301c97c62f5 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:19:07.472102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff24cb7-c174-467e-96a0-1d2364894827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:19:07.473738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff24cb7-c174-467e-96a0-1d2364894827 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:07.474188 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff24cb7-c174-467e-96a0-1d2364894827 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:07.474509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff24cb7-c174-467e-96a0-1d2364894827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:07.476678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6ff24cb7-c174-467e-96a0-1d2364894827 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:07.565811 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:19:07.607530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff24cb7-c174-467e-96a0-1d2364894827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:19:07.608990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff24cb7-c174-467e-96a0-1d2364894827 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:19:07.632720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 750/1484] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:19:09.704481 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2eee58-a166-4cba-8632-ec1dd53b9c2b None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:09.705402 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2eee58-a166-4cba-8632-ec1dd53b9c2b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:09.705402 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2eee58-a166-4cba-8632-ec1dd53b9c2b None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:09.706524 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 735/1485] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:19:09.714364 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217708b6-5914-4c10-a484-b98a7b8f27b5 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:09.714970 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217708b6-5914-4c10-a484-b98a7b8f27b5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:09.714970 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217708b6-5914-4c10-a484-b98a7b8f27b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:09.797837 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-217708b6-5914-4c10-a484-b98a7b8f27b5 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:19:09.799092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-217708b6-5914-4c10-a484-b98a7b8f27b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:09.915400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 751/1486] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:19:10.202815 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2eee58-a166-4cba-8632-ec1dd53b9c2b None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:19:10.204406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-da2eee58-a166-4cba-8632-ec1dd53b9c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:10.332931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2eee58-a166-4cba-8632-ec1dd53b9c2b None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:19:10.412843 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:19:10.414860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85801856-5bd0-4050-bcf6-f5dab271f257 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:19:10.415543 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85801856-5bd0-4050-bcf6-f5dab271f257 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:10.416090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85801856-5bd0-4050-bcf6-f5dab271f257 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:10.416623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85801856-5bd0-4050-bcf6-f5dab271f257 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:10.418578 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-85801856-5bd0-4050-bcf6-f5dab271f257 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:10.547017 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85801856-5bd0-4050-bcf6-f5dab271f257 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:19:10.548462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85801856-5bd0-4050-bcf6-f5dab271f257 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:19:10.569038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 736/1487] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:19:12.890535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cd09c-f0eb-4bd3-a0b4-bcf5bc12d816 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:12.891201 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cd09c-f0eb-4bd3-a0b4-bcf5bc12d816 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:12.891201 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cd09c-f0eb-4bd3-a0b4-bcf5bc12d816 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:12.892426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 752/1488] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:19:12.898987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7deef-35f7-409e-8edc-ff34f8e34414 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:12.899816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7deef-35f7-409e-8edc-ff34f8e34414 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:12.899978 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7deef-35f7-409e-8edc-ff34f8e34414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:12.975711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-40a7deef-35f7-409e-8edc-ff34f8e34414 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:19:12.976945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-40a7deef-35f7-409e-8edc-ff34f8e34414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:13.039186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:19:13.094227 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 737/1489] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:19:13.388263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cd09c-f0eb-4bd3-a0b4-bcf5bc12d816 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:19:13.389868 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9e2cd09c-f0eb-4bd3-a0b4-bcf5bc12d816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:13.430411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:19:13.515630 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cd09c-f0eb-4bd3-a0b4-bcf5bc12d816 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:19:13.608739 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049bab2-ccb3-4372-af01-c71b8a9d8930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:19:13.609515 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049bab2-ccb3-4372-af01-c71b8a9d8930 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:13.609937 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049bab2-ccb3-4372-af01-c71b8a9d8930 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:13.610168 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049bab2-ccb3-4372-af01-c71b8a9d8930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:13.612309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b049bab2-ccb3-4372-af01-c71b8a9d8930 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:13.713949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:19:13.752976 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049bab2-ccb3-4372-af01-c71b8a9d8930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:19:13.755105 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049bab2-ccb3-4372-af01-c71b8a9d8930 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:19:13.778580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 753/1490] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:19:15.998273 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d580b7-5b47-4b11-88e1-6b3a900c967d None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:15.998869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d580b7-5b47-4b11-88e1-6b3a900c967d None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:15.999220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d580b7-5b47-4b11-88e1-6b3a900c967d None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:16.001677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 738/1491] 10.222.0.88 () {54 vars in 927 bytes} [Tue Aug 23 22:19:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:19:16.008529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adebf84-e2c6-4435-95b9-0465ca494f46 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:16.009153 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adebf84-e2c6-4435-95b9-0465ca494f46 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:16.009153 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adebf84-e2c6-4435-95b9-0465ca494f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:16.085687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2adebf84-e2c6-4435-95b9-0465ca494f46 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:19:16.086836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2adebf84-e2c6-4435-95b9-0465ca494f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:16.190874 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 754/1492] 10.222.0.88 () {58 vars in 1024 bytes} [Tue Aug 23 22:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:19:16.491944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d580b7-5b47-4b11-88e1-6b3a900c967d None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:19:16.493519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-03d580b7-5b47-4b11-88e1-6b3a900c967d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:16.629262 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d580b7-5b47-4b11-88e1-6b3a900c967d None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:19:16.717110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b719cf9-f55c-4667-8cd0-d9b604970520 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:19:16.717714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b719cf9-f55c-4667-8cd0-d9b604970520 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:19:16.717990 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b719cf9-f55c-4667-8cd0-d9b604970520 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:19:16.718267 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b719cf9-f55c-4667-8cd0-d9b604970520 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:19:16.719870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8b719cf9-f55c-4667-8cd0-d9b604970520 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:19:16.731905 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:19:16.875265 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b719cf9-f55c-4667-8cd0-d9b604970520 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:19:16.876748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b719cf9-f55c-4667-8cd0-d9b604970520 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:19:16.903284 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 739/1493] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:19:26.188888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=82 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:19:26.900368 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=50 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:20:35.019009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83149d02-0ff2-431a-8fda-a97e0df250c9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:35.019654 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83149d02-0ff2-431a-8fda-a97e0df250c9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:35.019654 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83149d02-0ff2-431a-8fda-a97e0df250c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:35.117581 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-83149d02-0ff2-431a-8fda-a97e0df250c9 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:20:35.119328 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-83149d02-0ff2-431a-8fda-a97e0df250c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:20:35.233436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 755/1494] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:20:35.242467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c3263-4457-4695-9add-e7458f5c3fe3 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:35.243250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c3263-4457-4695-9add-e7458f5c3fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:35.243250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c3263-4457-4695-9add-e7458f5c3fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:35.336832 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-766c3263-4457-4695-9add-e7458f5c3fe3 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:20:35.338054 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-766c3263-4457-4695-9add-e7458f5c3fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:20:35.511933 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 740/1495] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:20:35.516930 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83149d02-0ff2-431a-8fda-a97e0df250c9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:20:35.518366 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-83149d02-0ff2-431a-8fda-a97e0df250c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:20:35.530459 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83149d02-0ff2-431a-8fda-a97e0df250c9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:20:35.606127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:20:35.652804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028a670-bd4e-4ac3-a822-98b79c7fe005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:20:35.653678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028a670-bd4e-4ac3-a822-98b79c7fe005 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:35.653954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028a670-bd4e-4ac3-a822-98b79c7fe005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:35.654231 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028a670-bd4e-4ac3-a822-98b79c7fe005 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:35.655048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028a670-bd4e-4ac3-a822-98b79c7fe005 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:20:35.655606 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028a670-bd4e-4ac3-a822-98b79c7fe005 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:20:35.656741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: WARNING py.warnings [None req-b028a670-bd4e-4ac3-a822-98b79c7fe005 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:20:35.656741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: warnings.warn(msg) Aug 23 22:20:35.656741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: Aug 23 22:20:35.657970 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028a670-bd4e-4ac3-a822-98b79c7fe005 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:20:35.679662 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 756/1496] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:20:35.692019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c3263-4457-4695-9add-e7458f5c3fe3 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:20:35.702275 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c3263-4457-4695-9add-e7458f5c3fe3 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:20:35.783368 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:20:35.822238 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7c9a9-2adb-4ab8-980d-957de34e8836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:20:35.822901 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7c9a9-2adb-4ab8-980d-957de34e8836 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:35.823437 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7c9a9-2adb-4ab8-980d-957de34e8836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:35.823790 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7c9a9-2adb-4ab8-980d-957de34e8836 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:35.824609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7c9a9-2adb-4ab8-980d-957de34e8836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:20:35.826989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: WARNING py.warnings [None req-f9a7c9a9-2adb-4ab8-980d-957de34e8836 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:20:35.826989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: warnings.warn(msg) Aug 23 22:20:35.826989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: Aug 23 22:20:35.828657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7c9a9-2adb-4ab8-980d-957de34e8836 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:20:35.891054 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 741/1497] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:20:35 2022] POST /identity/v3/projects => generated 377 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:20:35.896324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028a670-bd4e-4ac3-a822-98b79c7fe005 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:20:35.897945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b028a670-bd4e-4ac3-a822-98b79c7fe005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:20:35.935439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028a670-bd4e-4ac3-a822-98b79c7fe005 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:20:36.070939 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8bc03-0bb0-4129-bb3e-7455d7ea422a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:20:36.071710 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8bc03-0bb0-4129-bb3e-7455d7ea422a admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:36.071882 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8bc03-0bb0-4129-bb3e-7455d7ea422a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:36.072086 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8bc03-0bb0-4129-bb3e-7455d7ea422a admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:36.072734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8bc03-0bb0-4129-bb3e-7455d7ea422a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:20:36.073796 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: WARNING py.warnings [None req-c6b8bc03-0bb0-4129-bb3e-7455d7ea422a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:20:36.073796 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: warnings.warn(msg) Aug 23 22:20:36.073796 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: Aug 23 22:20:36.075054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8bc03-0bb0-4129-bb3e-7455d7ea422a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:20:36.124194 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 757/1498] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:20:35 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:20:36.130591 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7c9a9-2adb-4ab8-980d-957de34e8836 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:20:36.132165 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f9a7c9a9-2adb-4ab8-980d-957de34e8836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:20:36.210952 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7c9a9-2adb-4ab8-980d-957de34e8836 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:20:36.328570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:20:36.346998 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f8710-f4c3-40eb-b23e-776e24b0ca8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:20:36.347699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f8710-f4c3-40eb-b23e-776e24b0ca8d admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:36.347977 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f8710-f4c3-40eb-b23e-776e24b0ca8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:36.348257 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f8710-f4c3-40eb-b23e-776e24b0ca8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:36.348943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f8710-f4c3-40eb-b23e-776e24b0ca8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:20:36.350383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: WARNING py.warnings [None req-ed4f8710-f4c3-40eb-b23e-776e24b0ca8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:20:36.350383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: warnings.warn(msg) Aug 23 22:20:36.350383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: Aug 23 22:20:36.351319 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f8710-f4c3-40eb-b23e-776e24b0ca8d admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:20:36.367646 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 742/1499] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:20:36.372078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8bc03-0bb0-4129-bb3e-7455d7ea422a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:20:36.373728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c6b8bc03-0bb0-4129-bb3e-7455d7ea422a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:20:36.387642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8bc03-0bb0-4129-bb3e-7455d7ea422a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:20:36.425937 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:20:36.513965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1ad95-414b-4ce8-87b5-9c234785d069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:20:36.514494 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1ad95-414b-4ce8-87b5-9c234785d069 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:36.514788 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1ad95-414b-4ce8-87b5-9c234785d069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:36.515035 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1ad95-414b-4ce8-87b5-9c234785d069 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:36.515702 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1ad95-414b-4ce8-87b5-9c234785d069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:20:36.516732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: WARNING py.warnings [None req-f5b1ad95-414b-4ce8-87b5-9c234785d069 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:20:36.516732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: warnings.warn(msg) Aug 23 22:20:36.516732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: Aug 23 22:20:36.517687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1ad95-414b-4ce8-87b5-9c234785d069 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:20:36.535021 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 758/1500] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:20:36.539781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f8710-f4c3-40eb-b23e-776e24b0ca8d admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:20:36.541401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ed4f8710-f4c3-40eb-b23e-776e24b0ca8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:20:36.599659 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f8710-f4c3-40eb-b23e-776e24b0ca8d admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:20:36.744323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ea3d8-46a8-499b-abfe-3e5c065e70af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:20:36.745328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ea3d8-46a8-499b-abfe-3e5c065e70af admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:36.745754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ea3d8-46a8-499b-abfe-3e5c065e70af admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:36.746172 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ea3d8-46a8-499b-abfe-3e5c065e70af admin admin] PATH_INFO: `/v3/projects/59ecf0abb2fe43d29113156f64e902e0/users/e33b704c077d43a195884a0fba35214b/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:36.776847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ea3d8-46a8-499b-abfe-3e5c065e70af admin admin] RBAC: Authorizing `identity:create_grant(project_id=59ecf0abb2fe43d29113156f64e902e0, user_id=e33b704c077d43a195884a0fba35214b, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:20:36.777849 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: WARNING py.warnings [None req-0b1ea3d8-46a8-499b-abfe-3e5c065e70af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:20:36.777849 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: warnings.warn(msg) Aug 23 22:20:36.777849 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: Aug 23 22:20:36.779823 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ea3d8-46a8-499b-abfe-3e5c065e70af admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:20:36.808016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 743/1501] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:20:36 2022] PUT /identity/v3/projects/59ecf0abb2fe43d29113156f64e902e0/users/e33b704c077d43a195884a0fba35214b/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:20:36.821868 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffdc28-d2be-4efc-b044-656dd9f0b846 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:36.822205 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffdc28-d2be-4efc-b044-656dd9f0b846 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:36.822489 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffdc28-d2be-4efc-b044-656dd9f0b846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:36.889366 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-42ffdc28-d2be-4efc-b044-656dd9f0b846 None None] MFA Rules not processed for user `e33b704c077d43a195884a0fba35214b`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:20:36.890583 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-42ffdc28-d2be-4efc-b044-656dd9f0b846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:20:36.996900 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 759/1502] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:20:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:20:37.012061 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c04cc-0f38-46de-b8cf-7cee0589f8d2 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:37.012347 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c04cc-0f38-46de-b8cf-7cee0589f8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:37.012632 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c04cc-0f38-46de-b8cf-7cee0589f8d2 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:37.014133 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 744/1503] 10.222.0.88 () {54 vars in 964 bytes} [Tue Aug 23 22:20:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:20:37.020375 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155c222-2659-41da-ad4f-973363209e2b None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:37.020663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155c222-2659-41da-ad4f-973363209e2b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:37.020939 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155c222-2659-41da-ad4f-973363209e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:37.054344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:20:37.107780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b155c222-2659-41da-ad4f-973363209e2b None None] MFA Rules not processed for user `7d9ebd88431d46348089bc195491fca5`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:20:37.109004 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b155c222-2659-41da-ad4f-973363209e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:20:37.195445 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 760/1504] 10.222.0.88 () {58 vars in 1061 bytes} [Tue Aug 23 22:20:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:20:37.204010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c04cc-0f38-46de-b8cf-7cee0589f8d2 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:20:37.205668 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c60c04cc-0f38-46de-b8cf-7cee0589f8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:20:37.273200 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:20:37.348527 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c04cc-0f38-46de-b8cf-7cee0589f8d2 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:20:37.428871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc023e-c418-4070-954e-197dc33e7379 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:20:37.429518 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc023e-c418-4070-954e-197dc33e7379 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:37.429794 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc023e-c418-4070-954e-197dc33e7379 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:37.430063 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc023e-c418-4070-954e-197dc33e7379 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:37.431729 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8cdc023e-c418-4070-954e-197dc33e7379 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:20:37.574217 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc023e-c418-4070-954e-197dc33e7379 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:20:37.575843 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc023e-c418-4070-954e-197dc33e7379 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:20:37.579870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:20:37.598636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 745/1505] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:20:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:20:37.672504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dbb99-0212-4e62-99bf-5cab7184a9e1 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:37.673042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dbb99-0212-4e62-99bf-5cab7184a9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:37.673462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dbb99-0212-4e62-99bf-5cab7184a9e1 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:37.674929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 761/1506] 10.222.0.88 () {52 vars in 830 bytes} [Tue Aug 23 22:20:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:20:37.679563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc023e-c418-4070-954e-197dc33e7379 service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:20:37.680789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8cdc023e-c418-4070-954e-197dc33e7379 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:20:37.823895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc023e-c418-4070-954e-197dc33e7379 service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:20:37.965929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0eff1b-6146-4384-ba5f-c4e7359e8a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:20:37.966773 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eff1b-6146-4384-ba5f-c4e7359e8a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:37.967154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eff1b-6146-4384-ba5f-c4e7359e8a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:37.967540 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eff1b-6146-4384-ba5f-c4e7359e8a23 admin admin] PATH_INFO: `/v3/users/e33b704c077d43a195884a0fba35214b` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:37.987739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0eff1b-6146-4384-ba5f-c4e7359e8a23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e33b704c077d43a195884a0fba35214b)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:20:37.992290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: WARNING py.warnings [None req-0e0eff1b-6146-4384-ba5f-c4e7359e8a23 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:20:37.992290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: warnings.warn(msg) Aug 23 22:20:37.992290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: Aug 23 22:20:37.992290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0eff1b-6146-4384-ba5f-c4e7359e8a23 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:20:38.003250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:20:38.081087 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0e0eff1b-6146-4384-ba5f-c4e7359e8a23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e33b704c077d43a195884a0fba35214b'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:20:38.082367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0e0eff1b-6146-4384-ba5f-c4e7359e8a23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e33b704c077d43a195884a0fba35214b'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:20:38.107415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0e0eff1b-6146-4384-ba5f-c4e7359e8a23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e33b704c077d43a195884a0fba35214b'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:20:38.137214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0e0eff1b-6146-4384-ba5f-c4e7359e8a23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e33b704c077d43a195884a0fba35214b'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:20:38.148467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0e0eff1b-6146-4384-ba5f-c4e7359e8a23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e33b704c077d43a195884a0fba35214b'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:20:38.155118 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 746/1507] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:20:37 2022] DELETE /identity/v3/users/e33b704c077d43a195884a0fba35214b => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:20:38.161238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7e34b-9436-4308-b526-f8da0b423e9a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:38.161831 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7e34b-9436-4308-b526-f8da0b423e9a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:38.161970 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7e34b-9436-4308-b526-f8da0b423e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:38.237346 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b0f7e34b-9436-4308-b526-f8da0b423e9a None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:20:38.238580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b0f7e34b-9436-4308-b526-f8da0b423e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:20:38.337652 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:20:38.355919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 762/1508] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:20:38.366557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0eff1b-6146-4384-ba5f-c4e7359e8a23 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:20:38.368408 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0e0eff1b-6146-4384-ba5f-c4e7359e8a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:20:38.504825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0eff1b-6146-4384-ba5f-c4e7359e8a23 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:20:38.509760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25319b98-8509-46ca-b2ad-9b0eba88e795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:20:38.510418 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25319b98-8509-46ca-b2ad-9b0eba88e795 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:38.510725 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25319b98-8509-46ca-b2ad-9b0eba88e795 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:38.510972 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25319b98-8509-46ca-b2ad-9b0eba88e795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:38.513315 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-25319b98-8509-46ca-b2ad-9b0eba88e795 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:20:38.592625 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:20:38.621396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25319b98-8509-46ca-b2ad-9b0eba88e795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:20:38.622886 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25319b98-8509-46ca-b2ad-9b0eba88e795 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:20:38.657524 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 747/1509] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:20:39.077455 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7e34b-9436-4308-b526-f8da0b423e9a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:20:39.079251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b0f7e34b-9436-4308-b526-f8da0b423e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:20:39.092865 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7e34b-9436-4308-b526-f8da0b423e9a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:20:39.102689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a563321-fed3-4c96-8aae-eb81a7d1800c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:20:39.103540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a563321-fed3-4c96-8aae-eb81a7d1800c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:20:39.103963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a563321-fed3-4c96-8aae-eb81a7d1800c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:20:39.104241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a563321-fed3-4c96-8aae-eb81a7d1800c admin admin] PATH_INFO: `/v3/projects/59ecf0abb2fe43d29113156f64e902e0` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:20:39.135385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a563321-fed3-4c96-8aae-eb81a7d1800c admin admin] RBAC: Authorizing `identity:delete_project(project_id=59ecf0abb2fe43d29113156f64e902e0)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:20:39.138211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: WARNING py.warnings [None req-0a563321-fed3-4c96-8aae-eb81a7d1800c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:20:39.138211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: warnings.warn(msg) Aug 23 22:20:39.138211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: Aug 23 22:20:39.139500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a563321-fed3-4c96-8aae-eb81a7d1800c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:20:39.222675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0a563321-fed3-4c96-8aae-eb81a7d1800c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59ecf0abb2fe43d29113156f64e902e0', 'request_id': 'req-0a563321-fed3-4c96-8aae-eb81a7d1800c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:20:39.233365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0a563321-fed3-4c96-8aae-eb81a7d1800c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59ecf0abb2fe43d29113156f64e902e0', 'request_id': 'req-0a563321-fed3-4c96-8aae-eb81a7d1800c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:20:39.246731 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0a563321-fed3-4c96-8aae-eb81a7d1800c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59ecf0abb2fe43d29113156f64e902e0', 'request_id': 'req-0a563321-fed3-4c96-8aae-eb81a7d1800c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:20:39.271418 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0a563321-fed3-4c96-8aae-eb81a7d1800c admin admin] The token cache is being invalidate because project 59ecf0abb2fe43d29113156f64e902e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:20:39.272683 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0a563321-fed3-4c96-8aae-eb81a7d1800c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:20:39.272683 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0a563321-fed3-4c96-8aae-eb81a7d1800c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:20:39.274216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 763/1510] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:20:39 2022] DELETE /identity/v3/projects/59ecf0abb2fe43d29113156f64e902e0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:20:48.651669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=12 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:20:49.267171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:10.497114 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e9502-49cc-463e-9613-65ce9ab6af69 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:10.497805 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e9502-49cc-463e-9613-65ce9ab6af69 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:10.497805 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e9502-49cc-463e-9613-65ce9ab6af69 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:10.499076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:10.505037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e175b-b3cb-45aa-87dc-ba1f54812ebd None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:10.505627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e175b-b3cb-45aa-87dc-ba1f54812ebd None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:10.505627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e175b-b3cb-45aa-87dc-ba1f54812ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:10.569936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9d4e175b-b3cb-45aa-87dc-ba1f54812ebd None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:10.570890 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9d4e175b-b3cb-45aa-87dc-ba1f54812ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:10.632332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:22:10.792546 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e9502-49cc-463e-9613-65ce9ab6af69 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:10.794311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-1a9e9502-49cc-463e-9613-65ce9ab6af69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:10.908120 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e9502-49cc-463e-9613-65ce9ab6af69 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:10.914303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec115a33-5149-4e8f-980f-b71a6f1ad146 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:10.914924 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec115a33-5149-4e8f-980f-b71a6f1ad146 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:10.915208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec115a33-5149-4e8f-980f-b71a6f1ad146 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:10.915486 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec115a33-5149-4e8f-980f-b71a6f1ad146 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:10.925333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec115a33-5149-4e8f-980f-b71a6f1ad146 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:10.926941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec115a33-5149-4e8f-980f-b71a6f1ad146 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:10.937866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 749/1513] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:22:11.790615 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fef086-c0e9-4596-9a9e-d4cd73a12598 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:11.792069 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fef086-c0e9-4596-9a9e-d4cd73a12598 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:11.792069 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fef086-c0e9-4596-9a9e-d4cd73a12598 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:11.792856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:11.798454 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2fdbc-6569-4e6f-a80e-317d39600f45 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:11.798974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2fdbc-6569-4e6f-a80e-317d39600f45 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:11.799091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2fdbc-6569-4e6f-a80e-317d39600f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:11.826459 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-09c2fdbc-6569-4e6f-a80e-317d39600f45 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:11.827710 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-09c2fdbc-6569-4e6f-a80e-317d39600f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:11.836884 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:22:11.998139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fef086-c0e9-4596-9a9e-d4cd73a12598 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:12.018016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fef086-c0e9-4596-9a9e-d4cd73a12598 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:12.023734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc531a-96dc-46f3-a497-7530710d07ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:12.024352 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc531a-96dc-46f3-a497-7530710d07ff service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:12.024632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc531a-96dc-46f3-a497-7530710d07ff service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:12.024909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc531a-96dc-46f3-a497-7530710d07ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:12.034669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc531a-96dc-46f3-a497-7530710d07ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:12.036290 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc531a-96dc-46f3-a497-7530710d07ff service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:12.046961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 766/1516] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:22:13.057592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a480c66-0d36-4de5-b47b-7918dd36163a None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:13.058150 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a480c66-0d36-4de5-b47b-7918dd36163a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:13.058150 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a480c66-0d36-4de5-b47b-7918dd36163a None None] PATH_INFO: `/v3` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:13.059676 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 751/1517] 10.222.0.86 () {54 vars in 925 bytes} [Tue Aug 23 22:22:13 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:22:13.067164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d22a24-d5fa-4335-8b15-c6a859653cd2 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:13.067838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d22a24-d5fa-4335-8b15-c6a859653cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:13.067838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d22a24-d5fa-4335-8b15-c6a859653cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:13.142123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-32d22a24-d5fa-4335-8b15-c6a859653cd2 None None] MFA Rules not processed for user `e642dd7a07ca4afaa40c2769299d8bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:13.144569 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-32d22a24-d5fa-4335-8b15-c6a859653cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:13.234346 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 767/1518] 10.222.0.86 () {58 vars in 1015 bytes} [Tue Aug 23 22:22:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:22:21.817398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=49 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:23.231254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:23.276945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e127add-3f26-4ed9-99ea-4f0ecf053142 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:23.277528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e127add-3f26-4ed9-99ea-4f0ecf053142 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:23.277528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e127add-3f26-4ed9-99ea-4f0ecf053142 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:23.279031 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 752/1519] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:23.285448 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc18b7e-019c-4cdd-801b-c6e11d80ea17 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:23.285736 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc18b7e-019c-4cdd-801b-c6e11d80ea17 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:23.286020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc18b7e-019c-4cdd-801b-c6e11d80ea17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:23.308270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ebc18b7e-019c-4cdd-801b-c6e11d80ea17 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:23.309834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ebc18b7e-019c-4cdd-801b-c6e11d80ea17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:23.318925 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 768/1520] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:22:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:22:23.476735 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e127add-3f26-4ed9-99ea-4f0ecf053142 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:23.492129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e127add-3f26-4ed9-99ea-4f0ecf053142 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:23.498373 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a84a5-3fab-4320-9d80-a38ab06f1a2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:23.499227 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a84a5-3fab-4320-9d80-a38ab06f1a2d service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:23.499517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a84a5-3fab-4320-9d80-a38ab06f1a2d service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:23.499813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a84a5-3fab-4320-9d80-a38ab06f1a2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:23.508928 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a84a5-3fab-4320-9d80-a38ab06f1a2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:23.510548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a84a5-3fab-4320-9d80-a38ab06f1a2d service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:23.520534 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 753/1521] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:22:23.530982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2aeddd-3725-4e0f-b8f2-825022845e61 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:23.531171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2aeddd-3725-4e0f-b8f2-825022845e61 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:23.531448 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2aeddd-3725-4e0f-b8f2-825022845e61 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:23.533576 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 769/1522] 10.222.0.88 () {54 vars in 961 bytes} [Tue Aug 23 22:22:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:23.538961 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f71f22-e378-4dcf-bf9d-2133ec4030ca None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:23.539359 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f71f22-e378-4dcf-bf9d-2133ec4030ca None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:23.539505 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f71f22-e378-4dcf-bf9d-2133ec4030ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:23.560982 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-10f71f22-e378-4dcf-bf9d-2133ec4030ca None None] MFA Rules not processed for user `e642dd7a07ca4afaa40c2769299d8bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:23.562283 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-10f71f22-e378-4dcf-bf9d-2133ec4030ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:23.571463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 754/1523] 10.222.0.88 () {58 vars in 1058 bytes} [Tue Aug 23 22:22:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:22:23.579466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2aeddd-3725-4e0f-b8f2-825022845e61 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:23.593943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2aeddd-3725-4e0f-b8f2-825022845e61 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:23.599519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d9c1d-9bcc-4d6c-a659-7df0379338ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e642dd7a07ca4afaa40c2769299d8bad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:23.600142 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d9c1d-9bcc-4d6c-a659-7df0379338ca service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:23.600429 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d9c1d-9bcc-4d6c-a659-7df0379338ca service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:23.600702 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d9c1d-9bcc-4d6c-a659-7df0379338ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:23.609980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d9c1d-9bcc-4d6c-a659-7df0379338ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:23.612583 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d9c1d-9bcc-4d6c-a659-7df0379338ca service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:23.624283 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 770/1524] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:22:24.360845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0466cc-b33a-467e-90cb-3a242d0cc374 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:24.360845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0466cc-b33a-467e-90cb-3a242d0cc374 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:24.360845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0466cc-b33a-467e-90cb-3a242d0cc374 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:24.363034 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:22:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:24.368701 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61e738-35e5-4336-80cf-aab094b3f439 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:24.369400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61e738-35e5-4336-80cf-aab094b3f439 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:24.369400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61e738-35e5-4336-80cf-aab094b3f439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:24.390888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-cc61e738-35e5-4336-80cf-aab094b3f439 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:24.392164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-cc61e738-35e5-4336-80cf-aab094b3f439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:24.400357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:22:24.553579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0466cc-b33a-467e-90cb-3a242d0cc374 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:24.569072 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0466cc-b33a-467e-90cb-3a242d0cc374 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:24.574766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5fc6a-2bbc-4000-8789-5158736d9d52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:24.575386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5fc6a-2bbc-4000-8789-5158736d9d52 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:24.575668 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5fc6a-2bbc-4000-8789-5158736d9d52 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:24.575938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5fc6a-2bbc-4000-8789-5158736d9d52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:24.585491 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5fc6a-2bbc-4000-8789-5158736d9d52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:24.587079 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5fc6a-2bbc-4000-8789-5158736d9d52 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:24.597277 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 756/1527] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:22:28.515901 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe974d81-d1c0-4ff0-b4c4-3f22b8199827 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:28.516475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe974d81-d1c0-4ff0-b4c4-3f22b8199827 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:28.516475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe974d81-d1c0-4ff0-b4c4-3f22b8199827 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:28.518433 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:22:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:28.523895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4190576-d9ea-4331-a237-3598538e203a None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:28.524352 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4190576-d9ea-4331-a237-3598538e203a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:28.524463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4190576-d9ea-4331-a237-3598538e203a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:28.544716 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-b4190576-d9ea-4331-a237-3598538e203a None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:28.545964 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-b4190576-d9ea-4331-a237-3598538e203a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:28.554837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:22:28.847940 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe974d81-d1c0-4ff0-b4c4-3f22b8199827 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:28.849751 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fe974d81-d1c0-4ff0-b4c4-3f22b8199827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:28.909507 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe974d81-d1c0-4ff0-b4c4-3f22b8199827 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:28.913130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ddaf6-b89e-4838-815c-a81cf6059744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:28.913744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ddaf6-b89e-4838-815c-a81cf6059744 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:28.914031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ddaf6-b89e-4838-815c-a81cf6059744 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:28.914300 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ddaf6-b89e-4838-815c-a81cf6059744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:28.924480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ddaf6-b89e-4838-815c-a81cf6059744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:28.925965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ddaf6-b89e-4838-815c-a81cf6059744 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:28.936503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 773/1530] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:22:30.904965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421d688-5e81-4e62-8d3e-ff4db96f2940 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:30.905575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421d688-5e81-4e62-8d3e-ff4db96f2940 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:30.905575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421d688-5e81-4e62-8d3e-ff4db96f2940 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:30.906954 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:30.912604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca190fbb-1bbc-4af2-88f7-4eddef8531cd None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:30.913132 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca190fbb-1bbc-4af2-88f7-4eddef8531cd None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:30.913242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca190fbb-1bbc-4af2-88f7-4eddef8531cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:30.934355 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ca190fbb-1bbc-4af2-88f7-4eddef8531cd None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:30.935661 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ca190fbb-1bbc-4af2-88f7-4eddef8531cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:30.944690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:22:31.231194 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421d688-5e81-4e62-8d3e-ff4db96f2940 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:31.243186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421d688-5e81-4e62-8d3e-ff4db96f2940 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:31.246870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37562d45-5948-4b5b-822e-9ab097ba670b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:31.247504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37562d45-5948-4b5b-822e-9ab097ba670b service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:31.247782 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37562d45-5948-4b5b-822e-9ab097ba670b service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:31.248056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37562d45-5948-4b5b-822e-9ab097ba670b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:31.258302 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37562d45-5948-4b5b-822e-9ab097ba670b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:31.259750 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37562d45-5948-4b5b-822e-9ab097ba670b service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:31.270522 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 759/1533] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:22:34.820062 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2378af-efed-442a-8e0d-9c3f9bf1a3d6 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:34.820672 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2378af-efed-442a-8e0d-9c3f9bf1a3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:34.820672 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2378af-efed-442a-8e0d-9c3f9bf1a3d6 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:34.822008 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:34.827601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c26e7-35ab-4c20-9eab-a3d0d0ca06a1 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:34.828092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c26e7-35ab-4c20-9eab-a3d0d0ca06a1 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:34.828200 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c26e7-35ab-4c20-9eab-a3d0d0ca06a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:34.847939 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-c60c26e7-35ab-4c20-9eab-a3d0d0ca06a1 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:34.849264 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c60c26e7-35ab-4c20-9eab-a3d0d0ca06a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:34.857171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:22:35.416719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2378af-efed-442a-8e0d-9c3f9bf1a3d6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:35.419203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ad2378af-efed-442a-8e0d-9c3f9bf1a3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:35.505093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2378af-efed-442a-8e0d-9c3f9bf1a3d6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:35.508982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b9376-9732-4182-a01b-3427a0a63483 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:35.509620 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b9376-9732-4182-a01b-3427a0a63483 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:35.509902 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b9376-9732-4182-a01b-3427a0a63483 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:35.510174 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b9376-9732-4182-a01b-3427a0a63483 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:35.522047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b9376-9732-4182-a01b-3427a0a63483 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:35.523651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b9376-9732-4182-a01b-3427a0a63483 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:35.534718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 776/1536] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:22:37.523891 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b725f-0a50-49bf-a360-463f443c40f6 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:37.524592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b725f-0a50-49bf-a360-463f443c40f6 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:37.524592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b725f-0a50-49bf-a360-463f443c40f6 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:37.525851 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:37.531273 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f155de-be99-4e8c-8c48-2ca0cef16db8 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:37.531759 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f155de-be99-4e8c-8c48-2ca0cef16db8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:37.531881 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f155de-be99-4e8c-8c48-2ca0cef16db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:37.552783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-84f155de-be99-4e8c-8c48-2ca0cef16db8 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:37.554043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-84f155de-be99-4e8c-8c48-2ca0cef16db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:37.562378 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:22:38.126112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b725f-0a50-49bf-a360-463f443c40f6 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:38.128601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-2e5b725f-0a50-49bf-a360-463f443c40f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:38.141786 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b725f-0a50-49bf-a360-463f443c40f6 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:38.145727 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008cfe5-fb83-4df9-a3fa-e0759a11b3ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:38.146344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008cfe5-fb83-4df9-a3fa-e0759a11b3ee service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:38.146632 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008cfe5-fb83-4df9-a3fa-e0759a11b3ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:38.146901 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008cfe5-fb83-4df9-a3fa-e0759a11b3ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:38.156495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008cfe5-fb83-4df9-a3fa-e0759a11b3ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:38.158041 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008cfe5-fb83-4df9-a3fa-e0759a11b3ee service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:38.168155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 762/1539] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:22:47.547345 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=79 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:48.163688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=33 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:11.810413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10a65b-c730-4cb4-8679-12b4b23ad58e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:11.812228 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10a65b-c730-4cb4-8679-12b4b23ad58e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:11.812986 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10a65b-c730-4cb4-8679-12b4b23ad58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:11.861196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fa10a65b-c730-4cb4-8679-12b4b23ad58e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:11.863917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fa10a65b-c730-4cb4-8679-12b4b23ad58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:11.884791 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 778/1540] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:11.906412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca2688-c9d5-430c-bdeb-593f04cdc42e None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:11.907342 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca2688-c9d5-430c-bdeb-593f04cdc42e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:11.907660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca2688-c9d5-430c-bdeb-593f04cdc42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:11.966996 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-deca2688-c9d5-430c-bdeb-593f04cdc42e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:11.970788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-deca2688-c9d5-430c-bdeb-593f04cdc42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:11.993292 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 763/1541] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:12.002657 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10a65b-c730-4cb4-8679-12b4b23ad58e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:12.034826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10a65b-c730-4cb4-8679-12b4b23ad58e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:12.050757 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b87ade-6277-47ff-80d2-20e6127f7e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:12.052500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b87ade-6277-47ff-80d2-20e6127f7e21 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:12.053018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b87ade-6277-47ff-80d2-20e6127f7e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:12.053551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b87ade-6277-47ff-80d2-20e6127f7e21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:12.054789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b87ade-6277-47ff-80d2-20e6127f7e21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:24:12.055310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b87ade-6277-47ff-80d2-20e6127f7e21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:12.058667 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b87ade-6277-47ff-80d2-20e6127f7e21 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:12.100269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 779/1542] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:24:12.121772 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca2688-c9d5-430c-bdeb-593f04cdc42e None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:12.146092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca2688-c9d5-430c-bdeb-593f04cdc42e None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:12.165241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504792f1-26e0-4fd6-b936-44bb3a62f0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:12.166864 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504792f1-26e0-4fd6-b936-44bb3a62f0a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:12.167707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504792f1-26e0-4fd6-b936-44bb3a62f0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:12.168491 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504792f1-26e0-4fd6-b936-44bb3a62f0a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:12.170106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504792f1-26e0-4fd6-b936-44bb3a62f0a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:12.176148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504792f1-26e0-4fd6-b936-44bb3a62f0a9 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:12.284882 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 764/1543] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:12 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:12.291988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b87ade-6277-47ff-80d2-20e6127f7e21 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:12.321939 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b87ade-6277-47ff-80d2-20e6127f7e21 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:12.347592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895c020-224f-4fd5-9c90-ffc06541e088 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:12.348346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895c020-224f-4fd5-9c90-ffc06541e088 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:12.348856 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895c020-224f-4fd5-9c90-ffc06541e088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:12.423350 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-4895c020-224f-4fd5-9c90-ffc06541e088 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:12.425736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4895c020-224f-4fd5-9c90-ffc06541e088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:12.484858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3c54c-ef57-4ebb-a9d9-693dff8ed0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:12.485968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3c54c-ef57-4ebb-a9d9-693dff8ed0c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:12.486479 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3c54c-ef57-4ebb-a9d9-693dff8ed0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:12.486973 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3c54c-ef57-4ebb-a9d9-693dff8ed0c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:12.488218 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3c54c-ef57-4ebb-a9d9-693dff8ed0c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:12.492907 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3c54c-ef57-4ebb-a9d9-693dff8ed0c9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:12.558351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 780/1544] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:12 2022] POST /identity/v3/users => generated 294 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:12.560888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89e0e5-153b-4d27-9c4c-9ed46dca0cad None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:12.570019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89e0e5-153b-4d27-9c4c-9ed46dca0cad None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:12.570573 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89e0e5-153b-4d27-9c4c-9ed46dca0cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:12.628737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 765/1545] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:24:12.631537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a48b77-31cf-45c7-99ac-4ebed2404dc5 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:12.631992 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a48b77-31cf-45c7-99ac-4ebed2404dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:12.632521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a48b77-31cf-45c7-99ac-4ebed2404dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:12.641256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6a89e0e5-153b-4d27-9c4c-9ed46dca0cad None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:12.643347 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6a89e0e5-153b-4d27-9c4c-9ed46dca0cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:12.666070 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 781/1546] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:12.667407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89e0e5-153b-4d27-9c4c-9ed46dca0cad None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:12.682284 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-a4a48b77-31cf-45c7-99ac-4ebed2404dc5 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:12.693083 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a4a48b77-31cf-45c7-99ac-4ebed2404dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:12.693324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89e0e5-153b-4d27-9c4c-9ed46dca0cad None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:12.706900 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 766/1547] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:12.708508 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0673ee0-746e-493b-a7b1-011188a3ccd4 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:12.708838 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0673ee0-746e-493b-a7b1-011188a3ccd4 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:12.709148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0673ee0-746e-493b-a7b1-011188a3ccd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:12.720136 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932e4ce-95fd-4aea-90da-caf46c3591bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:12.720771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932e4ce-95fd-4aea-90da-caf46c3591bb admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:12.721049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932e4ce-95fd-4aea-90da-caf46c3591bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:12.721345 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932e4ce-95fd-4aea-90da-caf46c3591bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:12.722024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932e4ce-95fd-4aea-90da-caf46c3591bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:12.723842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932e4ce-95fd-4aea-90da-caf46c3591bb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:12.761756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 782/1548] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:12.768593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be13e30-7645-43e0-a68c-d451127eac81 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:12.768593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be13e30-7645-43e0-a68c-d451127eac81 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:12.768593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be13e30-7645-43e0-a68c-d451127eac81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:12.781091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-d0673ee0-746e-493b-a7b1-011188a3ccd4 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:12.786929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d0673ee0-746e-493b-a7b1-011188a3ccd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:12.798685 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 767/1549] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:12.798984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fd9f7-2bde-47ee-a016-a69c5eaa5ec3 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:12.799351 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fd9f7-2bde-47ee-a016-a69c5eaa5ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:12.799522 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fd9f7-2bde-47ee-a016-a69c5eaa5ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:12.825297 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-5be13e30-7645-43e0-a68c-d451127eac81 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:12.829384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-6c7fd9f7-2bde-47ee-a016-a69c5eaa5ec3 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:12.830471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5be13e30-7645-43e0-a68c-d451127eac81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:12.831051 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6c7fd9f7-2bde-47ee-a016-a69c5eaa5ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:12.840163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 768/1550] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:12.844854 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65a149-870b-4907-9496-c714f9203ad9 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:12.844854 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65a149-870b-4907-9496-c714f9203ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:12.844854 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65a149-870b-4907-9496-c714f9203ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:12.849424 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 783/1551] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:12.850653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be13e30-7645-43e0-a68c-d451127eac81 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:12.864073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be13e30-7645-43e0-a68c-d451127eac81 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:12.870761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665a31f-2317-45ea-bd4d-e69f0c73ec54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:12.871402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665a31f-2317-45ea-bd4d-e69f0c73ec54 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:12.871716 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665a31f-2317-45ea-bd4d-e69f0c73ec54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:12.871993 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665a31f-2317-45ea-bd4d-e69f0c73ec54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:12.872317 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-ee65a149-870b-4907-9496-c714f9203ad9 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:12.872696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665a31f-2317-45ea-bd4d-e69f0c73ec54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:12.874483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665a31f-2317-45ea-bd4d-e69f0c73ec54 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:12.875249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ee65a149-870b-4907-9496-c714f9203ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:12.887589 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 784/1552] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:12.888966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f2377-46fe-4419-b235-119283099e43 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:12.889812 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f2377-46fe-4419-b235-119283099e43 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:12.890186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f2377-46fe-4419-b235-119283099e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:12.897829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 769/1553] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:12.902190 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac6599-e63b-489c-9956-3944554102e6 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:12.902190 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac6599-e63b-489c-9956-3944554102e6 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:12.902190 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac6599-e63b-489c-9956-3944554102e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:12.915271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c10f2377-46fe-4419-b235-119283099e43 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:12.916610 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c10f2377-46fe-4419-b235-119283099e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:12.926797 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 785/1554] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:12.934557 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eded9-7c28-42d0-8567-b0ecd5ebc2be None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:12.934557 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eded9-7c28-42d0-8567-b0ecd5ebc2be None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:12.934557 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eded9-7c28-42d0-8567-b0ecd5ebc2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:12.951234 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-f0ac6599-e63b-489c-9956-3944554102e6 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:12.953388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f0ac6599-e63b-489c-9956-3944554102e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:12.965828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-982eded9-7c28-42d0-8567-b0ecd5ebc2be None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:12.970715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 770/1555] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:12.971009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-982eded9-7c28-42d0-8567-b0ecd5ebc2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:12.971203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac6599-e63b-489c-9956-3944554102e6 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:12.982276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 786/1556] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:12.984171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982eded9-7c28-42d0-8567-b0ecd5ebc2be None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:13.000167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982eded9-7c28-42d0-8567-b0ecd5ebc2be None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:13.000723 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac6599-e63b-489c-9956-3944554102e6 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:13.005987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff018b7d-8eae-4a76-b253-00ebf73a418c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:13.006927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff018b7d-8eae-4a76-b253-00ebf73a418c admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.007233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff018b7d-8eae-4a76-b253-00ebf73a418c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.007713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff018b7d-8eae-4a76-b253-00ebf73a418c admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.008429 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff018b7d-8eae-4a76-b253-00ebf73a418c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:24:13.008762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff018b7d-8eae-4a76-b253-00ebf73a418c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:13.010570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff018b7d-8eae-4a76-b253-00ebf73a418c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:13.015331 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8a496-2095-4ce3-b10d-cf5c48d42ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:13.017164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8a496-2095-4ce3-b10d-cf5c48d42ece admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.018216 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8a496-2095-4ce3-b10d-cf5c48d42ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.019428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8a496-2095-4ce3-b10d-cf5c48d42ece admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.022058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b8a496-2095-4ce3-b10d-cf5c48d42ece admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:24:13.023838 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b8a496-2095-4ce3-b10d-cf5c48d42ece admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:13.026500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: WARNING py.warnings [None req-77b8a496-2095-4ce3-b10d-cf5c48d42ece admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:24:13.026500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: warnings.warn(msg) Aug 23 22:24:13.026500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: Aug 23 22:24:13.037523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 787/1557] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:24:13.037523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff018b7d-8eae-4a76-b253-00ebf73a418c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:13.038010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b8a496-2095-4ce3-b10d-cf5c48d42ece admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:13.052211 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 771/1558] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:24:13.054429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8a496-2095-4ce3-b10d-cf5c48d42ece admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:13.055457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff018b7d-8eae-4a76-b253-00ebf73a418c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:13.068629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20369fd-6edf-43e1-9942-4cc8c34121de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:13.069841 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20369fd-6edf-43e1-9942-4cc8c34121de admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.071376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20369fd-6edf-43e1-9942-4cc8c34121de admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.072095 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20369fd-6edf-43e1-9942-4cc8c34121de admin admin] PATH_INFO: `/v3/projects/8be30316f0b745d783ec96bc983b2cb1/users/c06cd43d533f4621b0a69dbb0ef22dc4/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.078253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8a496-2095-4ce3-b10d-cf5c48d42ece admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:13.087405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02caeb6-9776-49c1-b6e3-617eafff8a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:13.089203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02caeb6-9776-49c1-b6e3-617eafff8a49 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.089203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02caeb6-9776-49c1-b6e3-617eafff8a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.089671 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02caeb6-9776-49c1-b6e3-617eafff8a49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.091192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02caeb6-9776-49c1-b6e3-617eafff8a49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:24:13.091674 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02caeb6-9776-49c1-b6e3-617eafff8a49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:13.095502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02caeb6-9776-49c1-b6e3-617eafff8a49 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:13.108742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20369fd-6edf-43e1-9942-4cc8c34121de admin admin] RBAC: Authorizing `identity:create_grant(project_id=8be30316f0b745d783ec96bc983b2cb1, user_id=c06cd43d533f4621b0a69dbb0ef22dc4, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:13.108742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: WARNING py.warnings [None req-e20369fd-6edf-43e1-9942-4cc8c34121de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:24:13.108742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: warnings.warn(msg) Aug 23 22:24:13.108742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: Aug 23 22:24:13.109301 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20369fd-6edf-43e1-9942-4cc8c34121de admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:13.123533 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 772/1559] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:24:13.127112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeca24f-f8ae-4a3c-9d97-9b426182a44d None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.128155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeca24f-f8ae-4a3c-9d97-9b426182a44d None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.128513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeca24f-f8ae-4a3c-9d97-9b426182a44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.137385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 788/1560] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:13 2022] PUT /identity/v3/projects/8be30316f0b745d783ec96bc983b2cb1/users/c06cd43d533f4621b0a69dbb0ef22dc4/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:13.138739 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20369fd-6edf-43e1-9942-4cc8c34121de admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:13.162816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-cfeca24f-f8ae-4a3c-9d97-9b426182a44d None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:13.165322 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-cfeca24f-f8ae-4a3c-9d97-9b426182a44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:13.230623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:13.263842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20369fd-6edf-43e1-9942-4cc8c34121de admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:13.267231 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 773/1561] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:13.269507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d97043-8e3b-4edc-b2c9-709aae7b63c7 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.270028 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d97043-8e3b-4edc-b2c9-709aae7b63c7 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.270223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eef9e3-7486-4c8b-b10c-3e2a40d674d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:13.270518 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d97043-8e3b-4edc-b2c9-709aae7b63c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.271147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eef9e3-7486-4c8b-b10c-3e2a40d674d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.271582 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eef9e3-7486-4c8b-b10c-3e2a40d674d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.271858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eef9e3-7486-4c8b-b10c-3e2a40d674d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.272817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eef9e3-7486-4c8b-b10c-3e2a40d674d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:24:13.273695 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eef9e3-7486-4c8b-b10c-3e2a40d674d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:13.275573 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eef9e3-7486-4c8b-b10c-3e2a40d674d5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:13.286645 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:13.291052 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 789/1562] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:24:13.292198 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eef9e3-7486-4c8b-b10c-3e2a40d674d5 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:13.310174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-09d97043-8e3b-4edc-b2c9-709aae7b63c7 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:13.310174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-09d97043-8e3b-4edc-b2c9-709aae7b63c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:13.315329 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eef9e3-7486-4c8b-b10c-3e2a40d674d5 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:13.321253 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13d17c-58c0-4d91-9234-15bb2d469fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:13.321981 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13d17c-58c0-4d91-9234-15bb2d469fac admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.322425 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13d17c-58c0-4d91-9234-15bb2d469fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.322712 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13d17c-58c0-4d91-9234-15bb2d469fac admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.323359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13d17c-58c0-4d91-9234-15bb2d469fac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:13.323778 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 774/1563] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:13.324456 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: WARNING py.warnings [None req-de13d17c-58c0-4d91-9234-15bb2d469fac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:24:13.324456 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: warnings.warn(msg) Aug 23 22:24:13.324456 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: Aug 23 22:24:13.325478 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d97043-8e3b-4edc-b2c9-709aae7b63c7 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:13.326185 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13d17c-58c0-4d91-9234-15bb2d469fac admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:13.347166 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d97043-8e3b-4edc-b2c9-709aae7b63c7 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:13.357428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367a18f-16c8-4ea8-a906-251c8a4c7e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:13.359379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367a18f-16c8-4ea8-a906-251c8a4c7e4e admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.359860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367a18f-16c8-4ea8-a906-251c8a4c7e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.360418 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367a18f-16c8-4ea8-a906-251c8a4c7e4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.361329 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367a18f-16c8-4ea8-a906-251c8a4c7e4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:13.362744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 790/1564] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:13 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:13.363904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367a18f-16c8-4ea8-a906-251c8a4c7e4e admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:13.364111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61413622-a7d8-4f83-9631-3a43acdb8182 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.364391 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61413622-a7d8-4f83-9631-3a43acdb8182 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.364668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61413622-a7d8-4f83-9631-3a43acdb8182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.390594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-61413622-a7d8-4f83-9631-3a43acdb8182 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:13.392042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-61413622-a7d8-4f83-9631-3a43acdb8182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:13.405291 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 775/1565] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:13 2022] POST /identity/v3/projects => generated 385 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:13.406568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367a18f-16c8-4ea8-a906-251c8a4c7e4e admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:13.481507 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 791/1566] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:13.485317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ab0f6-137e-48a1-9cfd-702e9c7bd016 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.485779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ab0f6-137e-48a1-9cfd-702e9c7bd016 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.486264 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ab0f6-137e-48a1-9cfd-702e9c7bd016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.515088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367a18f-16c8-4ea8-a906-251c8a4c7e4e admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:13.520395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f09f55-8a38-4220-ae3d-4fc2df381e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:13.521184 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f09f55-8a38-4220-ae3d-4fc2df381e2f admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.522916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-382ab0f6-137e-48a1-9cfd-702e9c7bd016 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:13.523163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f09f55-8a38-4220-ae3d-4fc2df381e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.523163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f09f55-8a38-4220-ae3d-4fc2df381e2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.523163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f09f55-8a38-4220-ae3d-4fc2df381e2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:13.523791 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-382ab0f6-137e-48a1-9cfd-702e9c7bd016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:13.524570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f09f55-8a38-4220-ae3d-4fc2df381e2f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:13.551071 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 792/1567] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:13.556824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ab0f6-137e-48a1-9cfd-702e9c7bd016 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:13.565160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 776/1568] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:13 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:13.566426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f09f55-8a38-4220-ae3d-4fc2df381e2f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:13.576054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ab0f6-137e-48a1-9cfd-702e9c7bd016 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:13.662161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7e694-ff4b-48dc-8ef6-d4f76a0afb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:13.663256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7e694-ff4b-48dc-8ef6-d4f76a0afb29 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.663751 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7e694-ff4b-48dc-8ef6-d4f76a0afb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.664249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7e694-ff4b-48dc-8ef6-d4f76a0afb29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.665520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7e694-ff4b-48dc-8ef6-d4f76a0afb29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:24:13.666042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7e694-ff4b-48dc-8ef6-d4f76a0afb29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:13.669223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7e694-ff4b-48dc-8ef6-d4f76a0afb29 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:13.679525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f09f55-8a38-4220-ae3d-4fc2df381e2f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:13.684792 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca888aa3-9fb4-4c75-8a3e-6a2fadb7d5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:13.685476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca888aa3-9fb4-4c75-8a3e-6a2fadb7d5ed admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.685749 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca888aa3-9fb4-4c75-8a3e-6a2fadb7d5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.686124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca888aa3-9fb4-4c75-8a3e-6a2fadb7d5ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.687319 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca888aa3-9fb4-4c75-8a3e-6a2fadb7d5ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:13.689371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca888aa3-9fb4-4c75-8a3e-6a2fadb7d5ed admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:13.695011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 793/1569] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:24:13.696381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7e694-ff4b-48dc-8ef6-d4f76a0afb29 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:13.714272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7e694-ff4b-48dc-8ef6-d4f76a0afb29 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:13.725372 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 777/1570] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:13 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:13.725372 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca888aa3-9fb4-4c75-8a3e-6a2fadb7d5ed admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:13.725728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3822c-114a-4d3c-83eb-623385a74f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:13.725728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3822c-114a-4d3c-83eb-623385a74f31 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.725728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3822c-114a-4d3c-83eb-623385a74f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.725728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3822c-114a-4d3c-83eb-623385a74f31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.726852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3822c-114a-4d3c-83eb-623385a74f31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:24:13.727354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3822c-114a-4d3c-83eb-623385a74f31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:13.730673 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3822c-114a-4d3c-83eb-623385a74f31 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:13.754748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 794/1571] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:24:13.754748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3822c-114a-4d3c-83eb-623385a74f31 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:13.772386 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3822c-114a-4d3c-83eb-623385a74f31 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:13.805800 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:13.817312 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:13.833488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca888aa3-9fb4-4c75-8a3e-6a2fadb7d5ed admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:13.839656 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad28491-edd8-4360-8534-8c11a27e6cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:13.840542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad28491-edd8-4360-8534-8c11a27e6cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.840987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad28491-edd8-4360-8534-8c11a27e6cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.841414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad28491-edd8-4360-8534-8c11a27e6cc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.842934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad28491-edd8-4360-8534-8c11a27e6cc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:13.844335 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: WARNING py.warnings [None req-8ad28491-edd8-4360-8534-8c11a27e6cc0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:24:13.844335 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: warnings.warn(msg) Aug 23 22:24:13.844335 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: Aug 23 22:24:13.846144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad28491-edd8-4360-8534-8c11a27e6cc0 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:13.861294 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243e31c-7362-4060-b1ea-e03e7aef44d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:13.862910 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243e31c-7362-4060-b1ea-e03e7aef44d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.865292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243e31c-7362-4060-b1ea-e03e7aef44d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.866396 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243e31c-7362-4060-b1ea-e03e7aef44d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.867270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243e31c-7362-4060-b1ea-e03e7aef44d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:13.870019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243e31c-7362-4060-b1ea-e03e7aef44d3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:13.885459 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 778/1572] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:13 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:13.885459 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88bbae-39a3-42df-947b-b80a14e5e4c2 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.885459 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88bbae-39a3-42df-947b-b80a14e5e4c2 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.885930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88bbae-39a3-42df-947b-b80a14e5e4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.912196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 795/1573] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:13 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:13.912196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243e31c-7362-4060-b1ea-e03e7aef44d3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:13.914691 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-8e88bbae-39a3-42df-947b-b80a14e5e4c2 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:13.916134 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8e88bbae-39a3-42df-947b-b80a14e5e4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:13.925563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243e31c-7362-4060-b1ea-e03e7aef44d3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:13.928146 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 779/1574] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:13.929315 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88bbae-39a3-42df-947b-b80a14e5e4c2 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:13.931156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8e88bbae-39a3-42df-947b-b80a14e5e4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:13.939544 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e47aa1-5c88-4e6b-b00b-d253e19f7f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:13.940521 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e47aa1-5c88-4e6b-b00b-d253e19f7f6c admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:13.940997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e47aa1-5c88-4e6b-b00b-d253e19f7f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:13.941424 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e47aa1-5c88-4e6b-b00b-d253e19f7f6c admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:13.942174 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e47aa1-5c88-4e6b-b00b-d253e19f7f6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:13.944637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e47aa1-5c88-4e6b-b00b-d253e19f7f6c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:13.986142 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 796/1575] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:13 2022] POST /identity/v3/users => generated 289 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:13.986142 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e47aa1-5c88-4e6b-b00b-d253e19f7f6c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:13.999384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e47aa1-5c88-4e6b-b00b-d253e19f7f6c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.006159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca97e0-af8c-4914-82a9-303abb66568a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.007133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca97e0-af8c-4914-82a9-303abb66568a admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.007437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca97e0-af8c-4914-82a9-303abb66568a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.008015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca97e0-af8c-4914-82a9-303abb66568a admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.008900 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca97e0-af8c-4914-82a9-303abb66568a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.011354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca97e0-af8c-4914-82a9-303abb66568a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.029560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88bbae-39a3-42df-947b-b80a14e5e4c2 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.035466 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89958113-b09c-4ec5-b296-5d5bf195bfc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.036339 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89958113-b09c-4ec5-b296-5d5bf195bfc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.036723 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89958113-b09c-4ec5-b296-5d5bf195bfc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.037315 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89958113-b09c-4ec5-b296-5d5bf195bfc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.049591 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89958113-b09c-4ec5-b296-5d5bf195bfc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.051129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89958113-b09c-4ec5-b296-5d5bf195bfc6 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.054617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 797/1576] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:13 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:14.054617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca97e0-af8c-4914-82a9-303abb66568a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.068309 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 780/1577] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:14.069269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89958113-b09c-4ec5-b296-5d5bf195bfc6 service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.083494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89958113-b09c-4ec5-b296-5d5bf195bfc6 service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.196094 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca97e0-af8c-4914-82a9-303abb66568a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.205836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cd013-d293-4038-b8aa-9c6224dc7212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.206938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cd013-d293-4038-b8aa-9c6224dc7212 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.207442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cd013-d293-4038-b8aa-9c6224dc7212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.207920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cd013-d293-4038-b8aa-9c6224dc7212 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.209405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013cd013-d293-4038-b8aa-9c6224dc7212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.212951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4a8d1-10e5-437e-a0f8-aa35163cc5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.213353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013cd013-d293-4038-b8aa-9c6224dc7212 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.214244 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4a8d1-10e5-437e-a0f8-aa35163cc5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.215021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4a8d1-10e5-437e-a0f8-aa35163cc5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.215614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4a8d1-10e5-437e-a0f8-aa35163cc5e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.216836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4a8d1-10e5-437e-a0f8-aa35163cc5e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.221460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4a8d1-10e5-437e-a0f8-aa35163cc5e7 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.257267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 798/1578] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:14 2022] POST /identity/v3/users => generated 287 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:14.257267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cd013-d293-4038-b8aa-9c6224dc7212 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.273452 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cd013-d293-4038-b8aa-9c6224dc7212 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.282271 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 781/1579] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:14 2022] POST /identity/v3/projects => generated 391 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:14.283297 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4a8d1-10e5-437e-a0f8-aa35163cc5e7 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.287937 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbb79f-8d02-4b9e-885f-d1958f791773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.291322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbb79f-8d02-4b9e-885f-d1958f791773 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.292152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbb79f-8d02-4b9e-885f-d1958f791773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.293654 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbb79f-8d02-4b9e-885f-d1958f791773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.294845 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbb79f-8d02-4b9e-885f-d1958f791773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.303145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbb79f-8d02-4b9e-885f-d1958f791773 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.329017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 799/1580] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:14.331936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbb79f-8d02-4b9e-885f-d1958f791773 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.351767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbb79f-8d02-4b9e-885f-d1958f791773 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.432962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:14.447388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4a8d1-10e5-437e-a0f8-aa35163cc5e7 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.456639 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780536a7-b52b-408f-b741-b4403e5dfd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.457637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780536a7-b52b-408f-b741-b4403e5dfd4b admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.458091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780536a7-b52b-408f-b741-b4403e5dfd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.458552 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780536a7-b52b-408f-b741-b4403e5dfd4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.459264 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780536a7-b52b-408f-b741-b4403e5dfd4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.461186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780536a7-b52b-408f-b741-b4403e5dfd4b admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.474220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01804ab1-60a2-47eb-bc1b-fdc081337e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.475487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01804ab1-60a2-47eb-bc1b-fdc081337e8a admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.475614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01804ab1-60a2-47eb-bc1b-fdc081337e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.476056 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01804ab1-60a2-47eb-bc1b-fdc081337e8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.477452 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 782/1581] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:14.477601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01804ab1-60a2-47eb-bc1b-fdc081337e8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:24:14.477840 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01804ab1-60a2-47eb-bc1b-fdc081337e8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.481530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780536a7-b52b-408f-b741-b4403e5dfd4b admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.481765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01804ab1-60a2-47eb-bc1b-fdc081337e8a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.485486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:14.504363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780536a7-b52b-408f-b741-b4403e5dfd4b admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.509162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 800/1582] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:24:14.510972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01804ab1-60a2-47eb-bc1b-fdc081337e8a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.513849 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac700a-93ff-432a-8eff-0cbd4144f3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.514836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac700a-93ff-432a-8eff-0cbd4144f3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.515293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac700a-93ff-432a-8eff-0cbd4144f3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.515734 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac700a-93ff-432a-8eff-0cbd4144f3e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.516816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac700a-93ff-432a-8eff-0cbd4144f3e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.519195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac700a-93ff-432a-8eff-0cbd4144f3e3 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.532170 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01804ab1-60a2-47eb-bc1b-fdc081337e8a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.535331 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 783/1583] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:14.536210 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac700a-93ff-432a-8eff-0cbd4144f3e3 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.555131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ed9b7-1e28-4c1a-a80e-0bae668d7e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.555926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ed9b7-1e28-4c1a-a80e-0bae668d7e8b admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.556211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ed9b7-1e28-4c1a-a80e-0bae668d7e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.556493 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ed9b7-1e28-4c1a-a80e-0bae668d7e8b admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.557252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ed9b7-1e28-4c1a-a80e-0bae668d7e8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.559019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac700a-93ff-432a-8eff-0cbd4144f3e3 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.559633 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ed9b7-1e28-4c1a-a80e-0bae668d7e8b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.568140 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8ff8c-e254-430a-ab0b-e593c809f25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.569514 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8ff8c-e254-430a-ab0b-e593c809f25f admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.570022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8ff8c-e254-430a-ab0b-e593c809f25f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.570520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8ff8c-e254-430a-ab0b-e593c809f25f admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.572091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8ff8c-e254-430a-ab0b-e593c809f25f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.579444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8ff8c-e254-430a-ab0b-e593c809f25f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.594280 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 801/1584] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:14 2022] POST /identity/v3/users => generated 289 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:14.595752 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ed9b7-1e28-4c1a-a80e-0bae668d7e8b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.613153 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 784/1585] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:14.615518 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8ff8c-e254-430a-ab0b-e593c809f25f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.627379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ed9b7-1e28-4c1a-a80e-0bae668d7e8b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.635987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8256482-8197-4ee0-acc9-cdc086dccabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.636395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8ff8c-e254-430a-ab0b-e593c809f25f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.637091 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8256482-8197-4ee0-acc9-cdc086dccabb admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.637773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8256482-8197-4ee0-acc9-cdc086dccabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.638366 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8256482-8197-4ee0-acc9-cdc086dccabb admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.642911 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8256482-8197-4ee0-acc9-cdc086dccabb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.646514 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8256482-8197-4ee0-acc9-cdc086dccabb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.647237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714eb09-1700-4c33-9966-0940e267462d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.648573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714eb09-1700-4c33-9966-0940e267462d admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.649444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714eb09-1700-4c33-9966-0940e267462d admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.649963 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714eb09-1700-4c33-9966-0940e267462d admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.651303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9714eb09-1700-4c33-9966-0940e267462d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.655863 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9714eb09-1700-4c33-9966-0940e267462d admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.685332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 802/1586] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:14 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:14.686999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8256482-8197-4ee0-acc9-cdc086dccabb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.700004 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8256482-8197-4ee0-acc9-cdc086dccabb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.706295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 785/1587] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:14.706295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714eb09-1700-4c33-9966-0940e267462d admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.712832 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e9da6-de8f-42ad-8ffb-c9f7440ce54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.714000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e9da6-de8f-42ad-8ffb-c9f7440ce54b admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.714553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e9da6-de8f-42ad-8ffb-c9f7440ce54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.715065 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e9da6-de8f-42ad-8ffb-c9f7440ce54b admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.716086 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e9da6-de8f-42ad-8ffb-c9f7440ce54b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.720039 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e9da6-de8f-42ad-8ffb-c9f7440ce54b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.727548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714eb09-1700-4c33-9966-0940e267462d admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.738847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 803/1588] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:14.740117 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7c2ba-991b-41f5-9ef7-6d70b7dec47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.741262 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7c2ba-991b-41f5-9ef7-6d70b7dec47a admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.741819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7c2ba-991b-41f5-9ef7-6d70b7dec47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.742455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7c2ba-991b-41f5-9ef7-6d70b7dec47a admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.743980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7c2ba-991b-41f5-9ef7-6d70b7dec47a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.744606 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e9da6-de8f-42ad-8ffb-c9f7440ce54b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.749586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7c2ba-991b-41f5-9ef7-6d70b7dec47a admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.758197 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e9da6-de8f-42ad-8ffb-c9f7440ce54b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.768933 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba02aa1-93d5-4e30-822c-ae320d001e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.769902 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba02aa1-93d5-4e30-822c-ae320d001e04 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.770341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba02aa1-93d5-4e30-822c-ae320d001e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.770993 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba02aa1-93d5-4e30-822c-ae320d001e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.771835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba02aa1-93d5-4e30-822c-ae320d001e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.774147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba02aa1-93d5-4e30-822c-ae320d001e04 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.789346 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 804/1589] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:14.790386 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba02aa1-93d5-4e30-822c-ae320d001e04 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.805312 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba02aa1-93d5-4e30-822c-ae320d001e04 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.812258 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af662de3-2d7d-4da3-bc9c-b0aeeccaa342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.813105 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af662de3-2d7d-4da3-bc9c-b0aeeccaa342 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.813515 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af662de3-2d7d-4da3-bc9c-b0aeeccaa342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.813863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af662de3-2d7d-4da3-bc9c-b0aeeccaa342 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.814658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af662de3-2d7d-4da3-bc9c-b0aeeccaa342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.816384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af662de3-2d7d-4da3-bc9c-b0aeeccaa342 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.824803 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 786/1590] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:14 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:14.824803 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7c2ba-991b-41f5-9ef7-6d70b7dec47a admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.833212 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 805/1591] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:14.833212 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af662de3-2d7d-4da3-bc9c-b0aeeccaa342 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.847042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af662de3-2d7d-4da3-bc9c-b0aeeccaa342 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.929989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b09a17-06e7-4e16-9d28-8ca9ed77a486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.931992 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b09a17-06e7-4e16-9d28-8ca9ed77a486 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.932595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b09a17-06e7-4e16-9d28-8ca9ed77a486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.933349 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b09a17-06e7-4e16-9d28-8ca9ed77a486 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.934228 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b09a17-06e7-4e16-9d28-8ca9ed77a486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.936892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b09a17-06e7-4e16-9d28-8ca9ed77a486 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.960511 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7c2ba-991b-41f5-9ef7-6d70b7dec47a admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.966623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 806/1592] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:14.966623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b09a17-06e7-4e16-9d28-8ca9ed77a486 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:14.971005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c872f2b-58eb-4d3f-855b-b6da7177f797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.972104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c872f2b-58eb-4d3f-855b-b6da7177f797 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.972525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c872f2b-58eb-4d3f-855b-b6da7177f797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.973182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c872f2b-58eb-4d3f-855b-b6da7177f797 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:14.974792 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c872f2b-58eb-4d3f-855b-b6da7177f797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:14.979169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c872f2b-58eb-4d3f-855b-b6da7177f797 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:14.984764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b09a17-06e7-4e16-9d28-8ca9ed77a486 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:14.993739 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0ed79-5c1d-4d9d-b4f6-52c671eb9ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:14.994783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0ed79-5c1d-4d9d-b4f6-52c671eb9ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:14.995250 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0ed79-5c1d-4d9d-b4f6-52c671eb9ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:14.995694 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0ed79-5c1d-4d9d-b4f6-52c671eb9ba8 admin admin] PATH_INFO: `/v3/projects/a5eed99762724bc69ca69883429093fc/users/e597a0bc1dd74a11be334d7e1cb18446/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:15.008479 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 787/1593] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:15.008479 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c872f2b-58eb-4d3f-855b-b6da7177f797 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:15.024892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0ed79-5c1d-4d9d-b4f6-52c671eb9ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5eed99762724bc69ca69883429093fc, user_id=e597a0bc1dd74a11be334d7e1cb18446, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:15.029163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c872f2b-58eb-4d3f-855b-b6da7177f797 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:15.030504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0ed79-5c1d-4d9d-b4f6-52c671eb9ba8 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:15.041815 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970664b2-8848-4faf-817a-21fb47a6c084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:15.043069 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970664b2-8848-4faf-817a-21fb47a6c084 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:15.044167 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970664b2-8848-4faf-817a-21fb47a6c084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:15.046722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970664b2-8848-4faf-817a-21fb47a6c084 admin admin] PATH_INFO: `/v3/projects/c0fc1c15f821495e9883814ffc7aed78/users/1db1a9a77be6452991ec4f99af814e7a/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:15.061820 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 807/1594] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:14 2022] PUT /identity/v3/projects/a5eed99762724bc69ca69883429093fc/users/e597a0bc1dd74a11be334d7e1cb18446/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:15.063937 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0ed79-5c1d-4d9d-b4f6-52c671eb9ba8 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:15.108710 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970664b2-8848-4faf-817a-21fb47a6c084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0fc1c15f821495e9883814ffc7aed78, user_id=1db1a9a77be6452991ec4f99af814e7a, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:15.111771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970664b2-8848-4faf-817a-21fb47a6c084 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:15.142356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 788/1595] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:15 2022] PUT /identity/v3/projects/c0fc1c15f821495e9883814ffc7aed78/users/1db1a9a77be6452991ec4f99af814e7a/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:15.143319 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970664b2-8848-4faf-817a-21fb47a6c084 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:15.204832 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:15.210043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0ed79-5c1d-4d9d-b4f6-52c671eb9ba8 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:15.315476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970664b2-8848-4faf-817a-21fb47a6c084 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:15.328752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4aca0-5f77-44dd-a1cb-3f32629e22ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:15.330159 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4aca0-5f77-44dd-a1cb-3f32629e22ca admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:15.330701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4aca0-5f77-44dd-a1cb-3f32629e22ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:15.331228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4aca0-5f77-44dd-a1cb-3f32629e22ca admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:15.332448 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4aca0-5f77-44dd-a1cb-3f32629e22ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:15.336662 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4aca0-5f77-44dd-a1cb-3f32629e22ca admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:15.380947 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:15.397166 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b622c0-4cf9-4fff-a376-85dc1032651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:15.397980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b622c0-4cf9-4fff-a376-85dc1032651f admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:15.398367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b622c0-4cf9-4fff-a376-85dc1032651f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:15.398665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b622c0-4cf9-4fff-a376-85dc1032651f admin admin] PATH_INFO: `/v3/projects/a297ef598b76404b86e9350e120b86c9/users/e720fd25d54b4dc6b3387ea762d7e4ae/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:15.399230 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 789/1596] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:15 2022] POST /identity/v3/users => generated 287 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:15.401464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4aca0-5f77-44dd-a1cb-3f32629e22ca admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:15.426191 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b622c0-4cf9-4fff-a376-85dc1032651f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a297ef598b76404b86e9350e120b86c9, user_id=e720fd25d54b4dc6b3387ea762d7e4ae, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:15.430214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b622c0-4cf9-4fff-a376-85dc1032651f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:15.430547 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4aca0-5f77-44dd-a1cb-3f32629e22ca admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:15.446458 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b840bd1-17fc-4404-b080-724c3d4ea52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:15.447298 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b840bd1-17fc-4404-b080-724c3d4ea52a admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:15.448334 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b840bd1-17fc-4404-b080-724c3d4ea52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:15.448842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b840bd1-17fc-4404-b080-724c3d4ea52a admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:15.450027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b840bd1-17fc-4404-b080-724c3d4ea52a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:15.452820 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b840bd1-17fc-4404-b080-724c3d4ea52a admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:15.456690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 808/1597] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:15 2022] PUT /identity/v3/projects/a297ef598b76404b86e9350e120b86c9/users/e720fd25d54b4dc6b3387ea762d7e4ae/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:15.458082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b622c0-4cf9-4fff-a376-85dc1032651f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:15.481529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 790/1598] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:15.482490 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b840bd1-17fc-4404-b080-724c3d4ea52a admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:15.503923 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b840bd1-17fc-4404-b080-724c3d4ea52a admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:15.624742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b622c0-4cf9-4fff-a376-85dc1032651f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:15.639815 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299d12f-ed4e-41e1-a2e8-d88d89b508ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:15.642092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299d12f-ed4e-41e1-a2e8-d88d89b508ba admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:15.643556 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299d12f-ed4e-41e1-a2e8-d88d89b508ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:15.644655 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299d12f-ed4e-41e1-a2e8-d88d89b508ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:15.647300 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299d12f-ed4e-41e1-a2e8-d88d89b508ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:15.651114 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299d12f-ed4e-41e1-a2e8-d88d89b508ba admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:15.654256 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeebf35-0338-47bf-9abd-3d1bf081b7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:15.655487 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeebf35-0338-47bf-9abd-3d1bf081b7bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:15.656163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeebf35-0338-47bf-9abd-3d1bf081b7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:15.656663 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeebf35-0338-47bf-9abd-3d1bf081b7bb admin admin] PATH_INFO: `/v3/projects/1728ea8aeac24483927690efa1de6818/users/2cc599a554fd45849c3c29910a74b9c0/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:15.694233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 809/1599] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:15.698083 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff79b4-1788-40e1-9847-8f15571a271c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:15.699885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff79b4-1788-40e1-9847-8f15571a271c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:15.700872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff79b4-1788-40e1-9847-8f15571a271c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:15.703355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeebf35-0338-47bf-9abd-3d1bf081b7bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1728ea8aeac24483927690efa1de6818, user_id=2cc599a554fd45849c3c29910a74b9c0, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:15.707466 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeebf35-0338-47bf-9abd-3d1bf081b7bb admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:15.741097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 791/1600] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:15 2022] PUT /identity/v3/projects/1728ea8aeac24483927690efa1de6818/users/2cc599a554fd45849c3c29910a74b9c0/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:15.741097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee0ec6-7212-4e84-a8bc-5a5937c7ae53 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:15.741097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee0ec6-7212-4e84-a8bc-5a5937c7ae53 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:15.741097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee0ec6-7212-4e84-a8bc-5a5937c7ae53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:15.775822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-41ff79b4-1788-40e1-9847-8f15571a271c None None] MFA Rules not processed for user `e597a0bc1dd74a11be334d7e1cb18446`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:15.777707 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-41ff79b4-1788-40e1-9847-8f15571a271c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:15.797434 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-a6ee0ec6-7212-4e84-a8bc-5a5937c7ae53 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:15.800304 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a6ee0ec6-7212-4e84-a8bc-5a5937c7ae53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:15.883947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 810/1601] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:15.883947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff79b4-1788-40e1-9847-8f15571a271c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:15.932967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:15.955696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 792/1602] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:15.957983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309088a-609c-4889-97d9-f3312c43924e None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:15.958603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309088a-609c-4889-97d9-f3312c43924e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:15.959119 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309088a-609c-4889-97d9-f3312c43924e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:15.989260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-9309088a-609c-4889-97d9-f3312c43924e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:15.991133 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-9309088a-609c-4889-97d9-f3312c43924e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:16.004327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 793/1603] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:16.005830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309088a-609c-4889-97d9-f3312c43924e None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:16.017469 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309088a-609c-4889-97d9-f3312c43924e None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:16.023171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e6fcb-aa1f-4367-bf55-8fa9e91964ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:16.024270 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e6fcb-aa1f-4367-bf55-8fa9e91964ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:16.024679 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e6fcb-aa1f-4367-bf55-8fa9e91964ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:16.025029 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e6fcb-aa1f-4367-bf55-8fa9e91964ef admin admin] PATH_INFO: `/v3/projects/398769c58d0d412aa45024c937820ba1/users/4e884c4f24db45a786618fc28d19bced/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:16.035073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff79b4-1788-40e1-9847-8f15571a271c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:16.041849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce150cda-11eb-429d-81bf-770aeecaae17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:16.043992 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce150cda-11eb-429d-81bf-770aeecaae17 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:16.044300 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce150cda-11eb-429d-81bf-770aeecaae17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:16.044659 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce150cda-11eb-429d-81bf-770aeecaae17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:16.045434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce150cda-11eb-429d-81bf-770aeecaae17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:16.047888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce150cda-11eb-429d-81bf-770aeecaae17 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:16.048591 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e6fcb-aa1f-4367-bf55-8fa9e91964ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=398769c58d0d412aa45024c937820ba1, user_id=4e884c4f24db45a786618fc28d19bced, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:16.051685 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e6fcb-aa1f-4367-bf55-8fa9e91964ef admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:16.067764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 811/1604] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:16.069116 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce150cda-11eb-429d-81bf-770aeecaae17 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:16.076414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 794/1605] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:16 2022] PUT /identity/v3/projects/398769c58d0d412aa45024c937820ba1/users/4e884c4f24db45a786618fc28d19bced/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:16.079967 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e72ded-5b64-4214-8f70-a6291f4aafde None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:16.080319 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e72ded-5b64-4214-8f70-a6291f4aafde None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:16.080740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e72ded-5b64-4214-8f70-a6291f4aafde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:16.090407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce150cda-11eb-429d-81bf-770aeecaae17 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:16.153734 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-67e72ded-5b64-4214-8f70-a6291f4aafde None None] MFA Rules not processed for user `2cc599a554fd45849c3c29910a74b9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:16.155703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-67e72ded-5b64-4214-8f70-a6291f4aafde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:16.190727 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9273b-c19a-477d-bfb2-dccf43347818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:16.191542 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9273b-c19a-477d-bfb2-dccf43347818 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:16.192173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9273b-c19a-477d-bfb2-dccf43347818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:16.192525 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9273b-c19a-477d-bfb2-dccf43347818 admin admin] PATH_INFO: `/v3/projects/1dfc84dc05d542d987194096f3b7e51a/users/78b05a538ce0466ea8b3e27a0a6fc162/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:16.219164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:16.221741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9273b-c19a-477d-bfb2-dccf43347818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dfc84dc05d542d987194096f3b7e51a, user_id=78b05a538ce0466ea8b3e27a0a6fc162, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:16.229463 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9273b-c19a-477d-bfb2-dccf43347818 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:16.272154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 812/1606] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:16 2022] PUT /identity/v3/projects/1dfc84dc05d542d987194096f3b7e51a/users/78b05a538ce0466ea8b3e27a0a6fc162/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:16.275264 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8bf1-e3dd-41a8-83bd-26836e6e9208 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:16.275264 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8bf1-e3dd-41a8-83bd-26836e6e9208 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:16.275264 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8bf1-e3dd-41a8-83bd-26836e6e9208 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:16.276306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 813/1607] 10.222.0.88 () {54 vars in 962 bytes} [Tue Aug 23 22:24:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:16.277343 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc8bf1-e3dd-41a8-83bd-26836e6e9208 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:16.306435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 795/1608] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:16.307811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e72ded-5b64-4214-8f70-a6291f4aafde None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:16.379611 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc8bf1-e3dd-41a8-83bd-26836e6e9208 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:16.385019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d85cbe-01e6-40e0-98ae-ef0ee08ab326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:16.385875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d85cbe-01e6-40e0-98ae-ef0ee08ab326 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:16.386257 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d85cbe-01e6-40e0-98ae-ef0ee08ab326 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:16.386543 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d85cbe-01e6-40e0-98ae-ef0ee08ab326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:16.437034 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e72ded-5b64-4214-8f70-a6291f4aafde None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:16.446089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a64f4-4437-4078-9406-4338b9aa8529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:16.448678 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a64f4-4437-4078-9406-4338b9aa8529 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:16.449231 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a64f4-4437-4078-9406-4338b9aa8529 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:16.449714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a64f4-4437-4078-9406-4338b9aa8529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:16.464841 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:16.493417 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d85cbe-01e6-40e0-98ae-ef0ee08ab326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:16.495137 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d85cbe-01e6-40e0-98ae-ef0ee08ab326 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:16.509547 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 814/1609] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:16.510757 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d85cbe-01e6-40e0-98ae-ef0ee08ab326 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:16.524612 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d85cbe-01e6-40e0-98ae-ef0ee08ab326 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:16.536794 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29904d-8f66-4718-b199-2bd165790f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:16.540413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29904d-8f66-4718-b199-2bd165790f4f admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:16.540703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29904d-8f66-4718-b199-2bd165790f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:16.541133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29904d-8f66-4718-b199-2bd165790f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:16.553179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29904d-8f66-4718-b199-2bd165790f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:16.553543 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a64f4-4437-4078-9406-4338b9aa8529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:16.558330 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29904d-8f66-4718-b199-2bd165790f4f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:16.563838 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a64f4-4437-4078-9406-4338b9aa8529 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:16.580731 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 796/1610] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:16.582357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c7b34-ba96-48b0-a818-22a950a7e5f4 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:16.582698 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c7b34-ba96-48b0-a818-22a950a7e5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:16.582952 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c7b34-ba96-48b0-a818-22a950a7e5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:16.595725 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 815/1611] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:16.601959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3140f6f-f92d-4937-ad02-ae8d388aa093 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:16.602308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3140f6f-f92d-4937-ad02-ae8d388aa093 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:16.602578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3140f6f-f92d-4937-ad02-ae8d388aa093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:16.612916 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-048c7b34-ba96-48b0-a818-22a950a7e5f4 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:16.614374 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-048c7b34-ba96-48b0-a818-22a950a7e5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:16.636249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 797/1612] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:16.640380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8142a-d561-4088-b2c9-9c57ea6d1740 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:16.640678 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8142a-d561-4088-b2c9-9c57ea6d1740 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:16.640968 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8142a-d561-4088-b2c9-9c57ea6d1740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:16.732637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d3140f6f-f92d-4937-ad02-ae8d388aa093 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:16.732637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d3140f6f-f92d-4937-ad02-ae8d388aa093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:16.732637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 816/1613] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:16.732637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3140f6f-f92d-4937-ad02-ae8d388aa093 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:16.742904 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3140f6f-f92d-4937-ad02-ae8d388aa093 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:16.744950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-0be8142a-d561-4088-b2c9-9c57ea6d1740 None None] MFA Rules not processed for user `e3517b3b077a4a1fa76163657b46930c`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:16.747553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0be8142a-d561-4088-b2c9-9c57ea6d1740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:16.752242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b643d-c47c-4780-b85a-047cbb332eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:16.753649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b643d-c47c-4780-b85a-047cbb332eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:16.754151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b643d-c47c-4780-b85a-047cbb332eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:16.756148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b643d-c47c-4780-b85a-047cbb332eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:16.756148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b643d-c47c-4780-b85a-047cbb332eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:16.760167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b643d-c47c-4780-b85a-047cbb332eb7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:16.849851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 817/1614] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:16 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:16.849851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b643d-c47c-4780-b85a-047cbb332eb7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:16.962616 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:16.997618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 798/1615] 10.222.0.88 () {58 vars in 1059 bytes} [Tue Aug 23 22:24:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:17.000034 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8142a-d561-4088-b2c9-9c57ea6d1740 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:17.021132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8142a-d561-4088-b2c9-9c57ea6d1740 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:17.033493 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b643d-c47c-4780-b85a-047cbb332eb7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:17.042058 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea797a-09f6-4092-9899-fd4bd87cc4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:17.042944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea797a-09f6-4092-9899-fd4bd87cc4c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:17.043460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea797a-09f6-4092-9899-fd4bd87cc4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:17.044101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea797a-09f6-4092-9899-fd4bd87cc4c9 admin admin] PATH_INFO: `/v3/projects/ab26978fb927482ba0da326c3aa6cbba/users/bc1090fafe494c159d0d24790b23b018/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:17.074098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea797a-09f6-4092-9899-fd4bd87cc4c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab26978fb927482ba0da326c3aa6cbba, user_id=bc1090fafe494c159d0d24790b23b018, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:17.081645 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea797a-09f6-4092-9899-fd4bd87cc4c9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:17.141339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 818/1616] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:16 2022] PUT /identity/v3/projects/ab26978fb927482ba0da326c3aa6cbba/users/bc1090fafe494c159d0d24790b23b018/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:17.141339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea797a-09f6-4092-9899-fd4bd87cc4c9 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:17.149344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb340be-96e3-47c7-9b08-e1e0e65a46ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:17.151593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb340be-96e3-47c7-9b08-e1e0e65a46ee service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:17.153097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb340be-96e3-47c7-9b08-e1e0e65a46ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:17.154273 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb340be-96e3-47c7-9b08-e1e0e65a46ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:17.193601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb340be-96e3-47c7-9b08-e1e0e65a46ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:17.196694 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb340be-96e3-47c7-9b08-e1e0e65a46ee service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:17.222564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 799/1617] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:17.238523 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb340be-96e3-47c7-9b08-e1e0e65a46ee service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:17.262928 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb340be-96e3-47c7-9b08-e1e0e65a46ee service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:17.294177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea797a-09f6-4092-9899-fd4bd87cc4c9 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:17.300701 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98600602-8d83-4d90-9ff7-5018f5b0dcea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:17.301573 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98600602-8d83-4d90-9ff7-5018f5b0dcea service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:17.302109 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98600602-8d83-4d90-9ff7-5018f5b0dcea service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:17.302735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98600602-8d83-4d90-9ff7-5018f5b0dcea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:17.396436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:17.489393 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98600602-8d83-4d90-9ff7-5018f5b0dcea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:17.492842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98600602-8d83-4d90-9ff7-5018f5b0dcea service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:17.499120 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c40c3-2cb1-4819-adfa-50c0216be97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:17.502041 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c40c3-2cb1-4819-adfa-50c0216be97a admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:17.502900 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c40c3-2cb1-4819-adfa-50c0216be97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:17.503378 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c40c3-2cb1-4819-adfa-50c0216be97a admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:17.506797 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c40c3-2cb1-4819-adfa-50c0216be97a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:17.513836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c40c3-2cb1-4819-adfa-50c0216be97a admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:17.534364 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 819/1618] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:17.539671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98600602-8d83-4d90-9ff7-5018f5b0dcea service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:17.593184 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 800/1619] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:17 2022] POST /identity/v3/users => generated 292 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:17.595036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b52952-f48f-45a1-8b9d-deb11edcef8f None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:17.598528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b52952-f48f-45a1-8b9d-deb11edcef8f None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:17.601724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b52952-f48f-45a1-8b9d-deb11edcef8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:17.704985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-d7b52952-f48f-45a1-8b9d-deb11edcef8f None None] MFA Rules not processed for user `bc1090fafe494c159d0d24790b23b018`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:17.709143 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d7b52952-f48f-45a1-8b9d-deb11edcef8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:17.916433 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98600602-8d83-4d90-9ff7-5018f5b0dcea service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:17.958644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9bce5f-dc16-4eb0-b637-332d50750324 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:17.962376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bce5f-dc16-4eb0-b637-332d50750324 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:17.964959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bce5f-dc16-4eb0-b637-332d50750324 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:17.966688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bce5f-dc16-4eb0-b637-332d50750324 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:18.048001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 801/1620] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:18.050101 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b52952-f48f-45a1-8b9d-deb11edcef8f None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:18.085872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b52952-f48f-45a1-8b9d-deb11edcef8f None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:18.101614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae218d70-e0b3-43bf-96d7-3f8977889554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:18.104829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae218d70-e0b3-43bf-96d7-3f8977889554 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:18.106385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae218d70-e0b3-43bf-96d7-3f8977889554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:18.106741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae218d70-e0b3-43bf-96d7-3f8977889554 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:18.107473 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae218d70-e0b3-43bf-96d7-3f8977889554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:18.117038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae218d70-e0b3-43bf-96d7-3f8977889554 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:18.158428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 802/1621] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:18.159187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702b7bf-57d8-4c58-9374-e625bfbc86d1 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:18.159187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702b7bf-57d8-4c58-9374-e625bfbc86d1 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:18.159187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702b7bf-57d8-4c58-9374-e625bfbc86d1 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:18.161641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 803/1622] 10.222.0.88 () {54 vars in 962 bytes} [Tue Aug 23 22:24:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:18.163684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702b7bf-57d8-4c58-9374-e625bfbc86d1 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:18.176862 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:18.191097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702b7bf-57d8-4c58-9374-e625bfbc86d1 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:18.197301 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6a658-14fb-4915-b4e6-f2557c200d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:18.198005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6a658-14fb-4915-b4e6-f2557c200d17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:18.198473 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6a658-14fb-4915-b4e6-f2557c200d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:18.198756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6a658-14fb-4915-b4e6-f2557c200d17 admin admin] PATH_INFO: `/v3/projects/58a593e57c15444bbbc1e7aab14d6eba/users/d9e4e2a6bcb048bab696c30cd74fd3a4/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:18.199482 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9bce5f-dc16-4eb0-b637-332d50750324 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:18.202340 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9bce5f-dc16-4eb0-b637-332d50750324 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:18.225722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6a658-14fb-4915-b4e6-f2557c200d17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a593e57c15444bbbc1e7aab14d6eba, user_id=d9e4e2a6bcb048bab696c30cd74fd3a4, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:18.228395 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 820/1623] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:18.229452 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6a658-14fb-4915-b4e6-f2557c200d17 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:18.230672 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2945c-fb55-46fa-938b-07662b6b3c57 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:18.231211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2945c-fb55-46fa-938b-07662b6b3c57 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:18.233925 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2945c-fb55-46fa-938b-07662b6b3c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:18.258767 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 804/1624] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:18 2022] PUT /identity/v3/projects/58a593e57c15444bbbc1e7aab14d6eba/users/d9e4e2a6bcb048bab696c30cd74fd3a4/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:18.262068 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6a658-14fb-4915-b4e6-f2557c200d17 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:18.285835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c8b2945c-fb55-46fa-938b-07662b6b3c57 None None] MFA Rules not processed for user `e3517b3b077a4a1fa76163657b46930c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:18.287536 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c8b2945c-fb55-46fa-938b-07662b6b3c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:18.392570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6a658-14fb-4915-b4e6-f2557c200d17 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:18.401883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc784422-888a-43b1-86a5-5da670d90bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:18.402582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc784422-888a-43b1-86a5-5da670d90bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:18.403088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc784422-888a-43b1-86a5-5da670d90bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:18.403377 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc784422-888a-43b1-86a5-5da670d90bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:18.405519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc784422-888a-43b1-86a5-5da670d90bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:18.409368 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc784422-888a-43b1-86a5-5da670d90bc5 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:18.429898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 805/1625] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:18.435706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc784422-888a-43b1-86a5-5da670d90bc5 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:18.457774 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc784422-888a-43b1-86a5-5da670d90bc5 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:18.466375 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe992bf-337b-4b56-91de-fff0303e0a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:18.471206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe992bf-337b-4b56-91de-fff0303e0a6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:18.471880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe992bf-337b-4b56-91de-fff0303e0a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:18.472373 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe992bf-337b-4b56-91de-fff0303e0a6a admin admin] PATH_INFO: `/v3/projects/58a593e57c15444bbbc1e7aab14d6eba/users/d9e4e2a6bcb048bab696c30cd74fd3a4/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:18.476031 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe992bf-337b-4b56-91de-fff0303e0a6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a593e57c15444bbbc1e7aab14d6eba, user_id=d9e4e2a6bcb048bab696c30cd74fd3a4, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:18.476401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 821/1626] 10.222.0.88 () {58 vars in 1059 bytes} [Tue Aug 23 22:24:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:18.479842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe992bf-337b-4b56-91de-fff0303e0a6a admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:18.485969 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2945c-fb55-46fa-938b-07662b6b3c57 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:18.497880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 806/1627] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:18 2022] PUT /identity/v3/projects/58a593e57c15444bbbc1e7aab14d6eba/users/d9e4e2a6bcb048bab696c30cd74fd3a4/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:18.510216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:18.511920 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88789282-75f6-4915-b573-fc3b0a0d2210 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:18.511920 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88789282-75f6-4915-b573-fc3b0a0d2210 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:18.513592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88789282-75f6-4915-b573-fc3b0a0d2210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:18.531030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2945c-fb55-46fa-938b-07662b6b3c57 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:18.653909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-88789282-75f6-4915-b573-fc3b0a0d2210 None None] MFA Rules not processed for user `d9e4e2a6bcb048bab696c30cd74fd3a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:18.656431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-88789282-75f6-4915-b573-fc3b0a0d2210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:18.817538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abda159-4d83-4925-a1c3-90ee7bc21eab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:18.818487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abda159-4d83-4925-a1c3-90ee7bc21eab service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:18.818815 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abda159-4d83-4925-a1c3-90ee7bc21eab service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:18.819135 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abda159-4d83-4925-a1c3-90ee7bc21eab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:18.941284 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 807/1628] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:18.965725 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88789282-75f6-4915-b573-fc3b0a0d2210 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:18.993161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abda159-4d83-4925-a1c3-90ee7bc21eab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:19.001389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abda159-4d83-4925-a1c3-90ee7bc21eab service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:19.053754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 822/1629] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:19.057094 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965efe38-eb3b-47d2-b943-d12686e8aad8 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:19.057554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965efe38-eb3b-47d2-b943-d12686e8aad8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:19.058238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965efe38-eb3b-47d2-b943-d12686e8aad8 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:19.060685 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 823/1630] 10.222.0.88 () {54 vars in 912 bytes} [Tue Aug 23 22:24:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:24:19.104056 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62585a-2720-4dd7-bb7d-73c7c56bf72f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:19.104550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62585a-2720-4dd7-bb7d-73c7c56bf72f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:19.105028 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62585a-2720-4dd7-bb7d-73c7c56bf72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:19.172269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0d62585a-2720-4dd7-bb7d-73c7c56bf72f None None] MFA Rules not processed for user `e3517b3b077a4a1fa76163657b46930c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:19.174156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0d62585a-2720-4dd7-bb7d-73c7c56bf72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:19.198313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:19.210210 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88789282-75f6-4915-b573-fc3b0a0d2210 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:19.213924 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 824/1631] 10.222.0.88 () {58 vars in 1009 bytes} [Tue Aug 23 22:24:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:24:19.226068 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20aacc-9e0a-43c6-935e-f70b43138a7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:19.226826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20aacc-9e0a-43c6-935e-f70b43138a7e service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:19.227112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20aacc-9e0a-43c6-935e-f70b43138a7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:19.227416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20aacc-9e0a-43c6-935e-f70b43138a7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:19.267745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20aacc-9e0a-43c6-935e-f70b43138a7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:19.269763 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20aacc-9e0a-43c6-935e-f70b43138a7e service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:19.277756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62585a-2720-4dd7-bb7d-73c7c56bf72f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:19.288548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 808/1632] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:19.327156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62585a-2720-4dd7-bb7d-73c7c56bf72f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:19.363688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25aca9-89b7-408b-ab11-582d1d3e925d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:19.364881 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25aca9-89b7-408b-ab11-582d1d3e925d service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:19.365423 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25aca9-89b7-408b-ab11-582d1d3e925d service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:19.365924 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25aca9-89b7-408b-ab11-582d1d3e925d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:19.402622 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25aca9-89b7-408b-ab11-582d1d3e925d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:19.407642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25aca9-89b7-408b-ab11-582d1d3e925d service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:19.434679 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 825/1633] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:24:20.039967 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb201fdf-5d67-47bc-83e3-79bd908e947b None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:20.040505 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb201fdf-5d67-47bc-83e3-79bd908e947b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:20.041508 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb201fdf-5d67-47bc-83e3-79bd908e947b None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:20.044260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 809/1634] 10.222.0.88 () {54 vars in 912 bytes} [Tue Aug 23 22:24:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:20.058650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76e98c-1d6a-4c2c-88b2-2a982428b687 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:20.059092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76e98c-1d6a-4c2c-88b2-2a982428b687 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:20.059589 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76e98c-1d6a-4c2c-88b2-2a982428b687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:20.187471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2d76e98c-1d6a-4c2c-88b2-2a982428b687 None None] MFA Rules not processed for user `e3517b3b077a4a1fa76163657b46930c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:20.195982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2d76e98c-1d6a-4c2c-88b2-2a982428b687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:20.260958 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 826/1635] 10.222.0.88 () {58 vars in 1009 bytes} [Tue Aug 23 22:24:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:20.326146 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb201fdf-5d67-47bc-83e3-79bd908e947b None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:20.369807 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb201fdf-5d67-47bc-83e3-79bd908e947b None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:20.408641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17593486-665c-46b9-a2f6-553314c7ccd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:20.409311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17593486-665c-46b9-a2f6-553314c7ccd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:20.409585 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17593486-665c-46b9-a2f6-553314c7ccd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:20.409960 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17593486-665c-46b9-a2f6-553314c7ccd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:20.456356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17593486-665c-46b9-a2f6-553314c7ccd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:20.458395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17593486-665c-46b9-a2f6-553314c7ccd0 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:20.508417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 810/1636] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:21.112976 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76e98c-1d6a-4c2c-88b2-2a982428b687 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:21.406696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76e98c-1d6a-4c2c-88b2-2a982428b687 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:21.428986 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e722fe-176b-4099-a4ae-83a5d1780b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:21.430342 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e722fe-176b-4099-a4ae-83a5d1780b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:21.430924 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e722fe-176b-4099-a4ae-83a5d1780b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:21.431215 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e722fe-176b-4099-a4ae-83a5d1780b7d admin admin] PATH_INFO: `/v3/users/2cc599a554fd45849c3c29910a74b9c0` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:21.436566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e722fe-176b-4099-a4ae-83a5d1780b7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cc599a554fd45849c3c29910a74b9c0)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:21.440746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: WARNING py.warnings [None req-f1e722fe-176b-4099-a4ae-83a5d1780b7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:24:21.440746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: warnings.warn(msg) Aug 23 22:24:21.440746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: Aug 23 22:24:21.448713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e722fe-176b-4099-a4ae-83a5d1780b7d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:21.477015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17593486-665c-46b9-a2f6-553314c7ccd0 service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:21.517228 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:21.543030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17593486-665c-46b9-a2f6-553314c7ccd0 service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:21.570620 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b7b18-6510-49cd-a3e7-d1a583f15abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:21.572123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b7b18-6510-49cd-a3e7-d1a583f15abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:21.573114 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b7b18-6510-49cd-a3e7-d1a583f15abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:21.573777 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b7b18-6510-49cd-a3e7-d1a583f15abc admin admin] PATH_INFO: `/v3/users/bc1090fafe494c159d0d24790b23b018` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:21.581840 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b7b18-6510-49cd-a3e7-d1a583f15abc admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc1090fafe494c159d0d24790b23b018)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:21.592737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b7b18-6510-49cd-a3e7-d1a583f15abc admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:21.671222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f1e722fe-176b-4099-a4ae-83a5d1780b7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cc599a554fd45849c3c29910a74b9c0'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:21.769247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f1e722fe-176b-4099-a4ae-83a5d1780b7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cc599a554fd45849c3c29910a74b9c0'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:21.798470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f1e722fe-176b-4099-a4ae-83a5d1780b7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cc599a554fd45849c3c29910a74b9c0'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:21.799906 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f1e722fe-176b-4099-a4ae-83a5d1780b7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cc599a554fd45849c3c29910a74b9c0'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:21.802086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-995b7b18-6510-49cd-a3e7-d1a583f15abc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc1090fafe494c159d0d24790b23b018'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:21.803322 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-995b7b18-6510-49cd-a3e7-d1a583f15abc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc1090fafe494c159d0d24790b23b018'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:21.833860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-995b7b18-6510-49cd-a3e7-d1a583f15abc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc1090fafe494c159d0d24790b23b018'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:21.836160 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f1e722fe-176b-4099-a4ae-83a5d1780b7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cc599a554fd45849c3c29910a74b9c0'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:21.856029 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 827/1637] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:24:21 2022] DELETE /identity/v3/users/2cc599a554fd45849c3c29910a74b9c0 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:21.862431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f12727-b71a-4cfd-9b3a-12a936303050 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:21.862744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f12727-b71a-4cfd-9b3a-12a936303050 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:21.863175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f12727-b71a-4cfd-9b3a-12a936303050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:21.868828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-995b7b18-6510-49cd-a3e7-d1a583f15abc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc1090fafe494c159d0d24790b23b018'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:21.884715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-995b7b18-6510-49cd-a3e7-d1a583f15abc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc1090fafe494c159d0d24790b23b018'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:21.904623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 811/1638] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:24:21 2022] DELETE /identity/v3/users/bc1090fafe494c159d0d24790b23b018 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:21.910045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aac4b4-9261-43e1-8950-4d841e042d4c None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:21.910647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aac4b4-9261-43e1-8950-4d841e042d4c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:21.910955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aac4b4-9261-43e1-8950-4d841e042d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:21.932292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e0f12727-b71a-4cfd-9b3a-12a936303050 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:21.934995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e0f12727-b71a-4cfd-9b3a-12a936303050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:21.966586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-e1aac4b4-9261-43e1-8950-4d841e042d4c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:21.968956 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e1aac4b4-9261-43e1-8950-4d841e042d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:22.169538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 828/1639] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:24:22.186209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 812/1640] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:22.205382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f12727-b71a-4cfd-9b3a-12a936303050 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:22.207448 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e0f12727-b71a-4cfd-9b3a-12a936303050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:22.228442 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aac4b4-9261-43e1-8950-4d841e042d4c None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:22.236303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e1aac4b4-9261-43e1-8950-4d841e042d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:22.398312 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f12727-b71a-4cfd-9b3a-12a936303050 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:22.403464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d5cd1-b39d-4d1b-95bd-1ab97d1a25fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:22.404216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d5cd1-b39d-4d1b-95bd-1ab97d1a25fd service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:22.404703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d5cd1-b39d-4d1b-95bd-1ab97d1a25fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:22.405309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d5cd1-b39d-4d1b-95bd-1ab97d1a25fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:22.419933 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d5cd1-b39d-4d1b-95bd-1ab97d1a25fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:22.423521 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d5cd1-b39d-4d1b-95bd-1ab97d1a25fd service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:22.441115 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 829/1641] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:24:22.463366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aac4b4-9261-43e1-8950-4d841e042d4c None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:22.476916 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd0105-43f6-42ff-a622-f971392e8785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:22.478481 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd0105-43f6-42ff-a622-f971392e8785 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:22.479430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd0105-43f6-42ff-a622-f971392e8785 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:22.479759 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd0105-43f6-42ff-a622-f971392e8785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:22.508727 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:22.519371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd0105-43f6-42ff-a622-f971392e8785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:22.521591 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd0105-43f6-42ff-a622-f971392e8785 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:22.550961 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 813/1642] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:27.285675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d5cd1-b39d-4d1b-95bd-1ab97d1a25fd service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:27.292320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ae8d5cd1-b39d-4d1b-95bd-1ab97d1a25fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:27.337512 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d5cd1-b39d-4d1b-95bd-1ab97d1a25fd service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:27.356632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a33cd-b723-4fbd-b8a3-e9442eee4887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:27.358277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a33cd-b723-4fbd-b8a3-e9442eee4887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:27.358775 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a33cd-b723-4fbd-b8a3-e9442eee4887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:27.360866 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a33cd-b723-4fbd-b8a3-e9442eee4887 admin admin] PATH_INFO: `/v3/projects/ab26978fb927482ba0da326c3aa6cbba` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:27.364061 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd0105-43f6-42ff-a622-f971392e8785 service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:27.366680 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8ffd0105-43f6-42ff-a622-f971392e8785 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:27.378195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a33cd-b723-4fbd-b8a3-e9442eee4887 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab26978fb927482ba0da326c3aa6cbba)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:27.393577 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a33cd-b723-4fbd-b8a3-e9442eee4887 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:27.409800 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd0105-43f6-42ff-a622-f971392e8785 service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:27.434315 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78268757-f0eb-4589-9a6d-c00f2a12b0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:27.436904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78268757-f0eb-4589-9a6d-c00f2a12b0c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:27.450532 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78268757-f0eb-4589-9a6d-c00f2a12b0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:27.453734 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78268757-f0eb-4589-9a6d-c00f2a12b0c6 admin admin] PATH_INFO: `/v3/projects/1728ea8aeac24483927690efa1de6818` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:27.457315 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78268757-f0eb-4589-9a6d-c00f2a12b0c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1728ea8aeac24483927690efa1de6818)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:27.464701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: WARNING py.warnings [None req-78268757-f0eb-4589-9a6d-c00f2a12b0c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:24:27.464701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: warnings.warn(msg) Aug 23 22:24:27.464701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: Aug 23 22:24:27.475049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78268757-f0eb-4589-9a6d-c00f2a12b0c6 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:27.550761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:27.590903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d31a33cd-b723-4fbd-b8a3-e9442eee4887 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab26978fb927482ba0da326c3aa6cbba', 'request_id': 'req-d31a33cd-b723-4fbd-b8a3-e9442eee4887'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:27.612848 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d31a33cd-b723-4fbd-b8a3-e9442eee4887 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab26978fb927482ba0da326c3aa6cbba', 'request_id': 'req-d31a33cd-b723-4fbd-b8a3-e9442eee4887'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:27.628183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d31a33cd-b723-4fbd-b8a3-e9442eee4887 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab26978fb927482ba0da326c3aa6cbba', 'request_id': 'req-d31a33cd-b723-4fbd-b8a3-e9442eee4887'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:27.668246 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d31a33cd-b723-4fbd-b8a3-e9442eee4887 admin admin] The token cache is being invalidate because project ab26978fb927482ba0da326c3aa6cbba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:24:27.673836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d31a33cd-b723-4fbd-b8a3-e9442eee4887 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:27.675896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d31a33cd-b723-4fbd-b8a3-e9442eee4887 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:27.681591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 830/1643] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:24:27 2022] DELETE /identity/v3/projects/ab26978fb927482ba0da326c3aa6cbba => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:27.689384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e4487-388e-4c4f-97dd-7474720c3027 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:27.689977 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e4487-388e-4c4f-97dd-7474720c3027 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:27.690443 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e4487-388e-4c4f-97dd-7474720c3027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:27.703665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-78268757-f0eb-4589-9a6d-c00f2a12b0c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1728ea8aeac24483927690efa1de6818', 'request_id': 'req-78268757-f0eb-4589-9a6d-c00f2a12b0c6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:27.807717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-78268757-f0eb-4589-9a6d-c00f2a12b0c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1728ea8aeac24483927690efa1de6818', 'request_id': 'req-78268757-f0eb-4589-9a6d-c00f2a12b0c6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:27.822588 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-cd7e4487-388e-4c4f-97dd-7474720c3027 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:27.822588 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-cd7e4487-388e-4c4f-97dd-7474720c3027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:27.860043 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-78268757-f0eb-4589-9a6d-c00f2a12b0c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1728ea8aeac24483927690efa1de6818', 'request_id': 'req-78268757-f0eb-4589-9a6d-c00f2a12b0c6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:27.894435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-78268757-f0eb-4589-9a6d-c00f2a12b0c6 admin admin] The token cache is being invalidate because project 1728ea8aeac24483927690efa1de6818 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:24:27.895289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-78268757-f0eb-4589-9a6d-c00f2a12b0c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:27.896468 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-78268757-f0eb-4589-9a6d-c00f2a12b0c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:27.901783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 814/1644] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:24:27 2022] DELETE /identity/v3/projects/1728ea8aeac24483927690efa1de6818 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:24:27.908501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78268757-f0eb-4589-9a6d-c00f2a12b0c6 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:27.913287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-78268757-f0eb-4589-9a6d-c00f2a12b0c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:28.015131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 831/1645] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:28.037520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2f398-8684-4428-8daa-657ccb95874e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:28.039047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2f398-8684-4428-8daa-657ccb95874e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:28.039628 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2f398-8684-4428-8daa-657ccb95874e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:28.103682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-afb2f398-8684-4428-8daa-657ccb95874e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:28.105316 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-afb2f398-8684-4428-8daa-657ccb95874e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:28.120369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 832/1646] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:28.126360 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2f398-8684-4428-8daa-657ccb95874e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:28.165354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2f398-8684-4428-8daa-657ccb95874e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:28.180427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78268757-f0eb-4589-9a6d-c00f2a12b0c6 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:28.187614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653efca6-ba3b-4a3b-89a7-f922435c73f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:28.188450 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653efca6-ba3b-4a3b-89a7-f922435c73f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:28.188836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653efca6-ba3b-4a3b-89a7-f922435c73f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:28.189355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653efca6-ba3b-4a3b-89a7-f922435c73f6 admin admin] PATH_INFO: `/v3/users/d9e4e2a6bcb048bab696c30cd74fd3a4` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:28.191111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653efca6-ba3b-4a3b-89a7-f922435c73f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9e4e2a6bcb048bab696c30cd74fd3a4)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:28.194749 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653efca6-ba3b-4a3b-89a7-f922435c73f6 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:28.200961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38c305-70b5-4e37-919b-7337ac0a17dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:28.204464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38c305-70b5-4e37-919b-7337ac0a17dd admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:28.204962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38c305-70b5-4e37-919b-7337ac0a17dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:28.215361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38c305-70b5-4e37-919b-7337ac0a17dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:28.218037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38c305-70b5-4e37-919b-7337ac0a17dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:24:28.218431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38c305-70b5-4e37-919b-7337ac0a17dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:28.221648 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38c305-70b5-4e37-919b-7337ac0a17dd admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:28.263350 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 833/1647] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:24:28.266421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38c305-70b5-4e37-919b-7337ac0a17dd admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:28.282204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-653efca6-ba3b-4a3b-89a7-f922435c73f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9e4e2a6bcb048bab696c30cd74fd3a4'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:28.283368 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-653efca6-ba3b-4a3b-89a7-f922435c73f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9e4e2a6bcb048bab696c30cd74fd3a4'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:28.298765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38c305-70b5-4e37-919b-7337ac0a17dd admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:28.299606 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-653efca6-ba3b-4a3b-89a7-f922435c73f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9e4e2a6bcb048bab696c30cd74fd3a4'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:28.311897 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4a3ee-4591-46dc-8e92-2d94d839e17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:28.313428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4a3ee-4591-46dc-8e92-2d94d839e17f admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:28.314202 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4a3ee-4591-46dc-8e92-2d94d839e17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:28.319748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4a3ee-4591-46dc-8e92-2d94d839e17f admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:28.319748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4a3ee-4591-46dc-8e92-2d94d839e17f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:28.323297 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4a3ee-4591-46dc-8e92-2d94d839e17f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:28.329789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-653efca6-ba3b-4a3b-89a7-f922435c73f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9e4e2a6bcb048bab696c30cd74fd3a4'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:28.337323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:28.350166 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-653efca6-ba3b-4a3b-89a7-f922435c73f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9e4e2a6bcb048bab696c30cd74fd3a4'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:28.377838 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 815/1648] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:24:27 2022] DELETE /identity/v3/users/d9e4e2a6bcb048bab696c30cd74fd3a4 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:28.433187 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 834/1649] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:28 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:28.435068 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653efca6-ba3b-4a3b-89a7-f922435c73f6 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:28.437114 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-653efca6-ba3b-4a3b-89a7-f922435c73f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:28.587429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653efca6-ba3b-4a3b-89a7-f922435c73f6 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:28.678533 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76dd00-0666-4c52-9c5f-606c97afe1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:28.679306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76dd00-0666-4c52-9c5f-606c97afe1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:28.679690 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76dd00-0666-4c52-9c5f-606c97afe1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:28.680069 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76dd00-0666-4c52-9c5f-606c97afe1b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:28.680847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76dd00-0666-4c52-9c5f-606c97afe1b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:28.685395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76dd00-0666-4c52-9c5f-606c97afe1b4 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:28.745633 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 816/1650] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:28 2022] POST /identity/v3/users => generated 293 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:24:28.752348 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4a3ee-4591-46dc-8e92-2d94d839e17f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:28.776540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4a3ee-4591-46dc-8e92-2d94d839e17f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:28.800650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e9f0b-e5cb-44c8-bcb5-debeebc9cc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:28.801296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e9f0b-e5cb-44c8-bcb5-debeebc9cc33 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:28.801580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e9f0b-e5cb-44c8-bcb5-debeebc9cc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:28.801852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e9f0b-e5cb-44c8-bcb5-debeebc9cc33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:28.802548 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e9f0b-e5cb-44c8-bcb5-debeebc9cc33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:28.804199 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e9f0b-e5cb-44c8-bcb5-debeebc9cc33 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:28.841823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 835/1651] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:28.844272 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76dd00-0666-4c52-9c5f-606c97afe1b4 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:28.880923 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76dd00-0666-4c52-9c5f-606c97afe1b4 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:28.906583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc67c98-a807-42de-8da3-3b326877eeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:28.908137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc67c98-a807-42de-8da3-3b326877eeac admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:28.908692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc67c98-a807-42de-8da3-3b326877eeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:28.909246 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc67c98-a807-42de-8da3-3b326877eeac admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:28.910703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc67c98-a807-42de-8da3-3b326877eeac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:28.914075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc67c98-a807-42de-8da3-3b326877eeac admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:28.953721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 817/1652] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:28.958069 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e9f0b-e5cb-44c8-bcb5-debeebc9cc33 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:28.973233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e9f0b-e5cb-44c8-bcb5-debeebc9cc33 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:28.994042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65404032-4cab-44a5-9803-949e296d0a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:28.994971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65404032-4cab-44a5-9803-949e296d0a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:28.995252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65404032-4cab-44a5-9803-949e296d0a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:28.995529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65404032-4cab-44a5-9803-949e296d0a43 admin admin] PATH_INFO: `/v3/projects/dc6d5bd146c84060bb02700eddd537a2/users/1cdc655825d74cdbbf708032ad551ec7/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:29.041148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65404032-4cab-44a5-9803-949e296d0a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc6d5bd146c84060bb02700eddd537a2, user_id=1cdc655825d74cdbbf708032ad551ec7, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:29.044485 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65404032-4cab-44a5-9803-949e296d0a43 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:29.083527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 836/1653] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:28 2022] PUT /identity/v3/projects/dc6d5bd146c84060bb02700eddd537a2/users/1cdc655825d74cdbbf708032ad551ec7/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:24:29.104626 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41749a91-3f43-4fc7-a11d-3aa6532b23f6 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:29.105205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41749a91-3f43-4fc7-a11d-3aa6532b23f6 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:29.105620 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41749a91-3f43-4fc7-a11d-3aa6532b23f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:29.189506 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-41749a91-3f43-4fc7-a11d-3aa6532b23f6 None None] MFA Rules not processed for user `1cdc655825d74cdbbf708032ad551ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:29.191751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-41749a91-3f43-4fc7-a11d-3aa6532b23f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:29.470372 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 818/1654] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:24:29.475445 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65404032-4cab-44a5-9803-949e296d0a43 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:29.671815 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:29.726433 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65404032-4cab-44a5-9803-949e296d0a43 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:29.771634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82304465-63bf-4e0b-969a-3c113619f13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:29.772809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82304465-63bf-4e0b-969a-3c113619f13f admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:29.773393 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82304465-63bf-4e0b-969a-3c113619f13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:29.775958 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82304465-63bf-4e0b-969a-3c113619f13f admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:29.778708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82304465-63bf-4e0b-969a-3c113619f13f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:29.793842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82304465-63bf-4e0b-969a-3c113619f13f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:29.878928 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 837/1655] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:29 2022] POST /identity/v3/projects => generated 385 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:29.881021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41749a91-3f43-4fc7-a11d-3aa6532b23f6 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:29.901873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41749a91-3f43-4fc7-a11d-3aa6532b23f6 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:29.913204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:30.026334 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc9709-f84f-4fcc-a1db-3be4080dae21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:30.027893 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc9709-f84f-4fcc-a1db-3be4080dae21 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:30.028567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc9709-f84f-4fcc-a1db-3be4080dae21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:30.029264 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc9709-f84f-4fcc-a1db-3be4080dae21 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:30.031031 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc9709-f84f-4fcc-a1db-3be4080dae21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:30.035280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc9709-f84f-4fcc-a1db-3be4080dae21 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:30.089326 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82304465-63bf-4e0b-969a-3c113619f13f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:30.095845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 819/1656] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:29 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:30.113241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82304465-63bf-4e0b-969a-3c113619f13f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:30.143642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced1979-24f8-4887-8f41-630ee7fd13c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:30.149124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced1979-24f8-4887-8f41-630ee7fd13c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:30.152242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced1979-24f8-4887-8f41-630ee7fd13c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:30.152597 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced1979-24f8-4887-8f41-630ee7fd13c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:30.153633 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced1979-24f8-4887-8f41-630ee7fd13c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:30.157838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced1979-24f8-4887-8f41-630ee7fd13c3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:30.180985 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 838/1657] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:24:30.187746 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc9709-f84f-4fcc-a1db-3be4080dae21 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:30.208714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc9709-f84f-4fcc-a1db-3be4080dae21 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:30.216446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c01c7-5210-4944-85f4-de0dbd207518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:30.218973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c01c7-5210-4944-85f4-de0dbd207518 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:30.219302 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c01c7-5210-4944-85f4-de0dbd207518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:30.219713 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c01c7-5210-4944-85f4-de0dbd207518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:30.221307 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c01c7-5210-4944-85f4-de0dbd207518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:30.224601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c01c7-5210-4944-85f4-de0dbd207518 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:30.249593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced1979-24f8-4887-8f41-630ee7fd13c3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:30.263093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 820/1658] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:30.279295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced1979-24f8-4887-8f41-630ee7fd13c3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:30.294680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc94b2a-9ea4-46ab-b72d-0c84cc606845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:30.297572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc94b2a-9ea4-46ab-b72d-0c84cc606845 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:30.299002 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc94b2a-9ea4-46ab-b72d-0c84cc606845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:30.300215 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc94b2a-9ea4-46ab-b72d-0c84cc606845 admin admin] PATH_INFO: `/v3/projects/97a176d63e3a40018d84dea69a60ec9d/users/25c4cbb30c58482690a388192ba8b8a7/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:30.328387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc94b2a-9ea4-46ab-b72d-0c84cc606845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a176d63e3a40018d84dea69a60ec9d, user_id=25c4cbb30c58482690a388192ba8b8a7, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:30.334123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc94b2a-9ea4-46ab-b72d-0c84cc606845 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:30.370077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 839/1659] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:30 2022] PUT /identity/v3/projects/97a176d63e3a40018d84dea69a60ec9d/users/25c4cbb30c58482690a388192ba8b8a7/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:30.393492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa05ad7-8827-4f4e-a5a0-b1b3d7999c68 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:30.394097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa05ad7-8827-4f4e-a5a0-b1b3d7999c68 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:30.394935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa05ad7-8827-4f4e-a5a0-b1b3d7999c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:30.453170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-8aa05ad7-8827-4f4e-a5a0-b1b3d7999c68 None None] MFA Rules not processed for user `25c4cbb30c58482690a388192ba8b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:30.455429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8aa05ad7-8827-4f4e-a5a0-b1b3d7999c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:30.761001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 821/1660] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:30.766356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc94b2a-9ea4-46ab-b72d-0c84cc606845 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:30.861115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa05ad7-8827-4f4e-a5a0-b1b3d7999c68 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:30.864172 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8aa05ad7-8827-4f4e-a5a0-b1b3d7999c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:31.261825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa05ad7-8827-4f4e-a5a0-b1b3d7999c68 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:31.289412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a49af-43c6-436d-8a8a-18884a1bbacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:31.289831 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc94b2a-9ea4-46ab-b72d-0c84cc606845 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:31.290467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a49af-43c6-436d-8a8a-18884a1bbacb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:31.290953 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a49af-43c6-436d-8a8a-18884a1bbacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:31.291439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a49af-43c6-436d-8a8a-18884a1bbacb admin admin] PATH_INFO: `/v3/projects/58a593e57c15444bbbc1e7aab14d6eba` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:31.298710 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a49af-43c6-436d-8a8a-18884a1bbacb admin admin] RBAC: Authorizing `identity:delete_project(project_id=58a593e57c15444bbbc1e7aab14d6eba)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:31.315434 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a49af-43c6-436d-8a8a-18884a1bbacb admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:31.374134 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b8454-2256-47e5-8535-54fac80ca133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:31.377089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b8454-2256-47e5-8535-54fac80ca133 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:31.379551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b8454-2256-47e5-8535-54fac80ca133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:31.386847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b8454-2256-47e5-8535-54fac80ca133 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:31.393808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b8454-2256-47e5-8535-54fac80ca133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:31.403740 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b8454-2256-47e5-8535-54fac80ca133 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:31.440105 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:31.533211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 840/1661] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:30 2022] POST /identity/v3/projects => generated 385 bytes in 768 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:31.538948 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b8454-2256-47e5-8535-54fac80ca133 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:31.542048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-311b8454-2256-47e5-8535-54fac80ca133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:31.567489 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-c00a49af-43c6-436d-8a8a-18884a1bbacb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58a593e57c15444bbbc1e7aab14d6eba', 'request_id': 'req-c00a49af-43c6-436d-8a8a-18884a1bbacb'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:31.602877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-c00a49af-43c6-436d-8a8a-18884a1bbacb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58a593e57c15444bbbc1e7aab14d6eba', 'request_id': 'req-c00a49af-43c6-436d-8a8a-18884a1bbacb'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:31.648837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-c00a49af-43c6-436d-8a8a-18884a1bbacb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58a593e57c15444bbbc1e7aab14d6eba', 'request_id': 'req-c00a49af-43c6-436d-8a8a-18884a1bbacb'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:31.695487 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-c00a49af-43c6-436d-8a8a-18884a1bbacb admin admin] The token cache is being invalidate because project 58a593e57c15444bbbc1e7aab14d6eba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:24:31.699343 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-c00a49af-43c6-436d-8a8a-18884a1bbacb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:31.700486 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-c00a49af-43c6-436d-8a8a-18884a1bbacb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:31.705958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 822/1662] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:24:30 2022] DELETE /identity/v3/projects/58a593e57c15444bbbc1e7aab14d6eba => generated 0 bytes in 846 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:31.709506 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a49af-43c6-436d-8a8a-18884a1bbacb admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:31.712048 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c00a49af-43c6-436d-8a8a-18884a1bbacb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:31.822620 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b8454-2256-47e5-8535-54fac80ca133 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:31.842721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee23862-e4df-4bce-8474-579d55642069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:31.847308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee23862-e4df-4bce-8474-579d55642069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:31.847820 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee23862-e4df-4bce-8474-579d55642069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:31.848323 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee23862-e4df-4bce-8474-579d55642069 admin admin] PATH_INFO: `/v3/users/e597a0bc1dd74a11be334d7e1cb18446` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:31.852607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee23862-e4df-4bce-8474-579d55642069 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e597a0bc1dd74a11be334d7e1cb18446)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:31.862245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee23862-e4df-4bce-8474-579d55642069 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:31.906503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:31.956722 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-aee23862-e4df-4bce-8474-579d55642069 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e597a0bc1dd74a11be334d7e1cb18446'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:31.976500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-aee23862-e4df-4bce-8474-579d55642069 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e597a0bc1dd74a11be334d7e1cb18446'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:31.994464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a49af-43c6-436d-8a8a-18884a1bbacb admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:31.996044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-aee23862-e4df-4bce-8474-579d55642069 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e597a0bc1dd74a11be334d7e1cb18446'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:31.997333 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-aee23862-e4df-4bce-8474-579d55642069 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e597a0bc1dd74a11be334d7e1cb18446'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:32.006384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b375b-89aa-4669-87ec-e71636f110b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:32.006384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b375b-89aa-4669-87ec-e71636f110b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:32.006384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b375b-89aa-4669-87ec-e71636f110b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:32.017554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b375b-89aa-4669-87ec-e71636f110b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:32.018405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3b375b-89aa-4669-87ec-e71636f110b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:32.020524 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3b375b-89aa-4669-87ec-e71636f110b2 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:32.025150 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-aee23862-e4df-4bce-8474-579d55642069 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e597a0bc1dd74a11be334d7e1cb18446'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:32.043207 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 841/1663] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:24:31 2022] DELETE /identity/v3/users/e597a0bc1dd74a11be334d7e1cb18446 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:32.049593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc84b98-da6c-4959-ab21-bf6ad689dc59 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:32.049868 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc84b98-da6c-4959-ab21-bf6ad689dc59 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:32.050254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc84b98-da6c-4959-ab21-bf6ad689dc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:32.089098 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 823/1664] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:31 2022] POST /identity/v3/users => generated 299 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:32.090475 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a795a-56af-44f2-9d20-394865d1acb9 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:32.090907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a795a-56af-44f2-9d20-394865d1acb9 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:32.091214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a795a-56af-44f2-9d20-394865d1acb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:32.104962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7dc84b98-da6c-4959-ab21-bf6ad689dc59 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:32.106843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7dc84b98-da6c-4959-ab21-bf6ad689dc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:32.148622 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-180a795a-56af-44f2-9d20-394865d1acb9 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:32.150627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-180a795a-56af-44f2-9d20-394865d1acb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:32.271256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 842/1665] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:32.291032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc84b98-da6c-4959-ab21-bf6ad689dc59 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:32.298655 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7dc84b98-da6c-4959-ab21-bf6ad689dc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:32.359965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc84b98-da6c-4959-ab21-bf6ad689dc59 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:32.363476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 824/1666] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:32.366314 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958093b-2cd6-42ec-98c3-41bb212d51c5 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:32.366901 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958093b-2cd6-42ec-98c3-41bb212d51c5 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:32.367662 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958093b-2cd6-42ec-98c3-41bb212d51c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:32.371686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d8227-330a-4da0-8d84-dd3fc1eb8b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:32.374909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d8227-330a-4da0-8d84-dd3fc1eb8b28 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:32.375772 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d8227-330a-4da0-8d84-dd3fc1eb8b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:32.376538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d8227-330a-4da0-8d84-dd3fc1eb8b28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:32.377605 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d8227-330a-4da0-8d84-dd3fc1eb8b28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:32.379840 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d8227-330a-4da0-8d84-dd3fc1eb8b28 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:32.400705 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 843/1667] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:32.407756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d8227-330a-4da0-8d84-dd3fc1eb8b28 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:32.413687 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-6958093b-2cd6-42ec-98c3-41bb212d51c5 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:32.416018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6958093b-2cd6-42ec-98c3-41bb212d51c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:32.432508 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d8227-330a-4da0-8d84-dd3fc1eb8b28 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:32.452161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dda73a-23bc-4654-b71c-3fc2f24b75dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:32.455018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 825/1668] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:32.462798 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dda73a-23bc-4654-b71c-3fc2f24b75dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:32.464811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958093b-2cd6-42ec-98c3-41bb212d51c5 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:32.465548 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dda73a-23bc-4654-b71c-3fc2f24b75dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:32.466282 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dda73a-23bc-4654-b71c-3fc2f24b75dd admin admin] PATH_INFO: `/v3/projects/53ef6b68509e4cc8a40442cf6c158f75/users/3420c8f7c8aa4daa8cbcdb4448157fb9/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:32.505310 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958093b-2cd6-42ec-98c3-41bb212d51c5 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:32.524099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1808f-fa94-4a9c-9954-a42d586655c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:32.526591 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1808f-fa94-4a9c-9954-a42d586655c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:32.528230 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1808f-fa94-4a9c-9954-a42d586655c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:32.530782 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dda73a-23bc-4654-b71c-3fc2f24b75dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ef6b68509e4cc8a40442cf6c158f75, user_id=3420c8f7c8aa4daa8cbcdb4448157fb9, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:32.531908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1808f-fa94-4a9c-9954-a42d586655c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:32.533462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1808f-fa94-4a9c-9954-a42d586655c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:24:32.534021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1808f-fa94-4a9c-9954-a42d586655c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:32.541405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dda73a-23bc-4654-b71c-3fc2f24b75dd admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:32.546209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1808f-fa94-4a9c-9954-a42d586655c4 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:32.593334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 844/1669] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:32 2022] PUT /identity/v3/projects/53ef6b68509e4cc8a40442cf6c158f75/users/3420c8f7c8aa4daa8cbcdb4448157fb9/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:24:32.599434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dda73a-23bc-4654-b71c-3fc2f24b75dd admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:32.602395 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-28dda73a-23bc-4654-b71c-3fc2f24b75dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:32.612558 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 826/1670] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:24:32.622906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1808f-fa94-4a9c-9954-a42d586655c4 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:32.669331 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1808f-fa94-4a9c-9954-a42d586655c4 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:32.739864 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:32.840583 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dda73a-23bc-4654-b71c-3fc2f24b75dd admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:32.851436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7da4c9-caf3-4f07-aea7-be2bce6f357c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:32.854303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7da4c9-caf3-4f07-aea7-be2bce6f357c service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:32.855109 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7da4c9-caf3-4f07-aea7-be2bce6f357c service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:32.856074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7da4c9-caf3-4f07-aea7-be2bce6f357c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:32.901631 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37691286-0aa2-4acb-89af-0ae16976769f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:32.902702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37691286-0aa2-4acb-89af-0ae16976769f admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:32.903171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37691286-0aa2-4acb-89af-0ae16976769f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:32.903640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37691286-0aa2-4acb-89af-0ae16976769f admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:32.906062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37691286-0aa2-4acb-89af-0ae16976769f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:32.908524 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37691286-0aa2-4acb-89af-0ae16976769f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:32.909287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:32.957002 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 827/1671] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:32.958588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37691286-0aa2-4acb-89af-0ae16976769f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:33.000235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37691286-0aa2-4acb-89af-0ae16976769f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:33.021871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef96ba-588b-4851-9adc-5c51d285ded3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:33.024379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef96ba-588b-4851-9adc-5c51d285ded3 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:33.024379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef96ba-588b-4851-9adc-5c51d285ded3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:33.024379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef96ba-588b-4851-9adc-5c51d285ded3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:33.024881 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef96ba-588b-4851-9adc-5c51d285ded3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:33.028819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef96ba-588b-4851-9adc-5c51d285ded3 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:33.033671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7da4c9-caf3-4f07-aea7-be2bce6f357c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:33.046438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7da4c9-caf3-4f07-aea7-be2bce6f357c service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:33.059608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 845/1672] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:33.061186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ab735-55bc-462a-b0b9-a7ad7573facf None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:33.061719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ab735-55bc-462a-b0b9-a7ad7573facf None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:33.073358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ab735-55bc-462a-b0b9-a7ad7573facf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:33.128887 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 828/1673] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:32 2022] POST /identity/v3/projects => generated 357 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:33.129433 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef96ba-588b-4851-9adc-5c51d285ded3 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:33.130396 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7f7ab735-55bc-462a-b0b9-a7ad7573facf None None] MFA Rules not processed for user `c06cd43d533f4621b0a69dbb0ef22dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:33.131947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7f7ab735-55bc-462a-b0b9-a7ad7573facf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:33.316849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 846/1674] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:33.316849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ab735-55bc-462a-b0b9-a7ad7573facf None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:33.368032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef96ba-588b-4851-9adc-5c51d285ded3 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:33.388286 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911825d5-29bb-4702-b337-0172ad876927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:33.390074 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911825d5-29bb-4702-b337-0172ad876927 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:33.390566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911825d5-29bb-4702-b337-0172ad876927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:33.391062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911825d5-29bb-4702-b337-0172ad876927 admin admin] PATH_INFO: `/v3/projects/53ef6b68509e4cc8a40442cf6c158f75/users/3420c8f7c8aa4daa8cbcdb4448157fb9/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:33.402312 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911825d5-29bb-4702-b337-0172ad876927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ef6b68509e4cc8a40442cf6c158f75, user_id=3420c8f7c8aa4daa8cbcdb4448157fb9, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:33.408568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911825d5-29bb-4702-b337-0172ad876927 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:33.455229 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 829/1675] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:33 2022] PUT /identity/v3/projects/53ef6b68509e4cc8a40442cf6c158f75/users/3420c8f7c8aa4daa8cbcdb4448157fb9/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:33.455229 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911825d5-29bb-4702-b337-0172ad876927 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:33.462258 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-911825d5-29bb-4702-b337-0172ad876927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:33.470340 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ab735-55bc-462a-b0b9-a7ad7573facf None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:33.595772 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:33.615184 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c6962-ee73-4403-8346-9fba993667da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:33.616244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c6962-ee73-4403-8346-9fba993667da admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:33.616523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c6962-ee73-4403-8346-9fba993667da admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:33.616806 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c6962-ee73-4403-8346-9fba993667da admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:33.617509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c6962-ee73-4403-8346-9fba993667da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:33.619681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c6962-ee73-4403-8346-9fba993667da admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:33.678815 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 847/1676] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:33 2022] POST /identity/v3/users => generated 279 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:33.680445 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00281977-a5a5-48c9-96f3-5772e82dda61 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:33.681016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00281977-a5a5-48c9-96f3-5772e82dda61 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:33.681587 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00281977-a5a5-48c9-96f3-5772e82dda61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:33.756627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-00281977-a5a5-48c9-96f3-5772e82dda61 None None] MFA Rules not processed for user `3420c8f7c8aa4daa8cbcdb4448157fb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:33.759894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-00281977-a5a5-48c9-96f3-5772e82dda61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:33.763161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911825d5-29bb-4702-b337-0172ad876927 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:33.790182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80aeb8-0cec-48ab-9dbe-bb3f74ec8ff7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:33.794923 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80aeb8-0cec-48ab-9dbe-bb3f74ec8ff7 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:33.797418 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80aeb8-0cec-48ab-9dbe-bb3f74ec8ff7 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:33.798306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80aeb8-0cec-48ab-9dbe-bb3f74ec8ff7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:33.942399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 848/1677] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:33.944047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00281977-a5a5-48c9-96f3-5772e82dda61 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:33.964421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:33.971201 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00281977-a5a5-48c9-96f3-5772e82dda61 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:33.994931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cd32d-c850-4031-8ec7-515120387689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:33.995620 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cd32d-c850-4031-8ec7-515120387689 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:33.995912 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cd32d-c850-4031-8ec7-515120387689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:33.996189 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cd32d-c850-4031-8ec7-515120387689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:33.996894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062cd32d-c850-4031-8ec7-515120387689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:33.999257 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062cd32d-c850-4031-8ec7-515120387689 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:34.048004 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 849/1678] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:34.051912 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cd32d-c850-4031-8ec7-515120387689 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:34.056551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-062cd32d-c850-4031-8ec7-515120387689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:34.125367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80aeb8-0cec-48ab-9dbe-bb3f74ec8ff7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:34.133426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80aeb8-0cec-48ab-9dbe-bb3f74ec8ff7 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:34.172594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 830/1679] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 730 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:34.176422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80aeb8-0cec-48ab-9dbe-bb3f74ec8ff7 service nova] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:34.215286 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80aeb8-0cec-48ab-9dbe-bb3f74ec8ff7 service nova] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:34.241618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abdcd7-0465-4d11-8e03-76b2db28d3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:34.244737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abdcd7-0465-4d11-8e03-76b2db28d3b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:34.247909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abdcd7-0465-4d11-8e03-76b2db28d3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:34.249439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abdcd7-0465-4d11-8e03-76b2db28d3b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:34.252061 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3abdcd7-0465-4d11-8e03-76b2db28d3b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:34.261146 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3abdcd7-0465-4d11-8e03-76b2db28d3b6 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:34.264338 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cd32d-c850-4031-8ec7-515120387689 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:34.280616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e31d41-f79c-4a8f-b80f-b8d321c3effe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:34.281736 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e31d41-f79c-4a8f-b80f-b8d321c3effe service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:34.284900 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e31d41-f79c-4a8f-b80f-b8d321c3effe service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:34.285972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e31d41-f79c-4a8f-b80f-b8d321c3effe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:34.312518 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 831/1680] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:34.314356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abdcd7-0465-4d11-8e03-76b2db28d3b6 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:34.336143 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e31d41-f79c-4a8f-b80f-b8d321c3effe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:34.339171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e31d41-f79c-4a8f-b80f-b8d321c3effe service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:34.363408 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abdcd7-0465-4d11-8e03-76b2db28d3b6 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:34.388013 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb91e77-6c7f-4fd6-a3ac-526712877d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:34.389772 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb91e77-6c7f-4fd6-a3ac-526712877d9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:34.390274 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb91e77-6c7f-4fd6-a3ac-526712877d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:34.390764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb91e77-6c7f-4fd6-a3ac-526712877d9a admin admin] PATH_INFO: `/v3/projects/df2149b69f854ab2a2e094e51148b3b6/users/573b3fe429ac4f5c99918fa610770f57/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:34.395854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 850/1681] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:34.471858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb91e77-6c7f-4fd6-a3ac-526712877d9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=df2149b69f854ab2a2e094e51148b3b6, user_id=573b3fe429ac4f5c99918fa610770f57, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:34.489937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb91e77-6c7f-4fd6-a3ac-526712877d9a admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:34.540830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 832/1682] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:34 2022] PUT /identity/v3/projects/df2149b69f854ab2a2e094e51148b3b6/users/573b3fe429ac4f5c99918fa610770f57/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:34.562481 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd737c1-d2ce-487f-b66b-f7be1046352f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:34.563401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd737c1-d2ce-487f-b66b-f7be1046352f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:34.563995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd737c1-d2ce-487f-b66b-f7be1046352f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:34.643268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7fd737c1-d2ce-487f-b66b-f7be1046352f None None] MFA Rules not processed for user `573b3fe429ac4f5c99918fa610770f57`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:34.647638 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7fd737c1-d2ce-487f-b66b-f7be1046352f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:34.887110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:35.006253 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 851/1683] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:35.006987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb91e77-6c7f-4fd6-a3ac-526712877d9a admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:35.365939 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb91e77-6c7f-4fd6-a3ac-526712877d9a admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:35.410959 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e832654-1eed-4b4a-a12a-24ab2a9680c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:35.416141 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e832654-1eed-4b4a-a12a-24ab2a9680c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:35.417788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e832654-1eed-4b4a-a12a-24ab2a9680c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:35.421630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e832654-1eed-4b4a-a12a-24ab2a9680c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:35.431768 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e832654-1eed-4b4a-a12a-24ab2a9680c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:35.444631 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e832654-1eed-4b4a-a12a-24ab2a9680c0 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:35.643139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd737c1-d2ce-487f-b66b-f7be1046352f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:35.658162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 833/1684] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:35 2022] POST /identity/v3/projects => generated 355 bytes in 628 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:35.695493 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd737c1-d2ce-487f-b66b-f7be1046352f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:35.887771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e832654-1eed-4b4a-a12a-24ab2a9680c0 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:35.890450 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-9e832654-1eed-4b4a-a12a-24ab2a9680c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:35.938002 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e175e-245b-43b7-a83e-0ffe929ed762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:35.938724 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e175e-245b-43b7-a83e-0ffe929ed762 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:35.939012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e175e-245b-43b7-a83e-0ffe929ed762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:35.939297 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e175e-245b-43b7-a83e-0ffe929ed762 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:35.939945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e175e-245b-43b7-a83e-0ffe929ed762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:35.952325 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e175e-245b-43b7-a83e-0ffe929ed762 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:36.026469 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e832654-1eed-4b4a-a12a-24ab2a9680c0 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:36.031761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 852/1685] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:35 2022] POST /identity/v3/users => generated 284 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:36.037031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e175e-245b-43b7-a83e-0ffe929ed762 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:36.047998 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac08e2-d58a-4f7f-95b6-0401dc91ff52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:36.049886 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac08e2-d58a-4f7f-95b6-0401dc91ff52 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:36.050354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac08e2-d58a-4f7f-95b6-0401dc91ff52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:36.051826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac08e2-d58a-4f7f-95b6-0401dc91ff52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:36.058323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-75ac08e2-d58a-4f7f-95b6-0401dc91ff52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:36.084083 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e175e-245b-43b7-a83e-0ffe929ed762 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:36.114380 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7f188-d2b5-4a94-93a1-7762e496784e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:36.115759 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7f188-d2b5-4a94-93a1-7762e496784e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:36.116454 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7f188-d2b5-4a94-93a1-7762e496784e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:36.117705 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7f188-d2b5-4a94-93a1-7762e496784e admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:36.119209 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7f188-d2b5-4a94-93a1-7762e496784e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:36.122740 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7f188-d2b5-4a94-93a1-7762e496784e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:36.165735 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:36.166013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 853/1686] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:36.171435 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7f188-d2b5-4a94-93a1-7762e496784e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:36.201547 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7f188-d2b5-4a94-93a1-7762e496784e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:36.208352 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a94fa-448c-4257-86bc-e8e600e13eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:36.209302 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a94fa-448c-4257-86bc-e8e600e13eeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:36.209600 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a94fa-448c-4257-86bc-e8e600e13eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:36.209879 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a94fa-448c-4257-86bc-e8e600e13eeb admin admin] PATH_INFO: `/v3/projects/01a3ab7444bb440189417eb5dcedf204/users/b4ca49c8a2cf4575a326df65ef5d0bd4/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:36.249999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a94fa-448c-4257-86bc-e8e600e13eeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=01a3ab7444bb440189417eb5dcedf204, user_id=b4ca49c8a2cf4575a326df65ef5d0bd4, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:36.253310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a94fa-448c-4257-86bc-e8e600e13eeb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:36.285961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 854/1687] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:36 2022] PUT /identity/v3/projects/01a3ab7444bb440189417eb5dcedf204/users/b4ca49c8a2cf4575a326df65ef5d0bd4/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:36.291945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a94fa-448c-4257-86bc-e8e600e13eeb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:36.299495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac08e2-d58a-4f7f-95b6-0401dc91ff52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:36.302431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac08e2-d58a-4f7f-95b6-0401dc91ff52 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:36.333178 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 834/1688] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:24:36.340177 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac08e2-d58a-4f7f-95b6-0401dc91ff52 service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:36.349514 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-75ac08e2-d58a-4f7f-95b6-0401dc91ff52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:36.397587 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac08e2-d58a-4f7f-95b6-0401dc91ff52 service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:36.452024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a94fa-448c-4257-86bc-e8e600e13eeb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:36.467367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025101d2-a97d-44fa-84c5-60b46264d313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:36.468128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025101d2-a97d-44fa-84c5-60b46264d313 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:36.468413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025101d2-a97d-44fa-84c5-60b46264d313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:36.468704 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025101d2-a97d-44fa-84c5-60b46264d313 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:36.469414 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025101d2-a97d-44fa-84c5-60b46264d313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:36.471280 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025101d2-a97d-44fa-84c5-60b46264d313 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:36.503827 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 855/1689] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:24:36.507586 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025101d2-a97d-44fa-84c5-60b46264d313 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:36.535102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025101d2-a97d-44fa-84c5-60b46264d313 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:36.548002 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b7e80-4ddb-4103-be65-3b433d901fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:36.549003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b7e80-4ddb-4103-be65-3b433d901fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:36.549447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b7e80-4ddb-4103-be65-3b433d901fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:36.549960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b7e80-4ddb-4103-be65-3b433d901fdd admin admin] PATH_INFO: `/v3/projects/01a3ab7444bb440189417eb5dcedf204/users/b4ca49c8a2cf4575a326df65ef5d0bd4/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:36.554481 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b7e80-4ddb-4103-be65-3b433d901fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=01a3ab7444bb440189417eb5dcedf204, user_id=b4ca49c8a2cf4575a326df65ef5d0bd4, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:36.562270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b7e80-4ddb-4103-be65-3b433d901fdd admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:36.604919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 856/1690] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:36 2022] PUT /identity/v3/projects/01a3ab7444bb440189417eb5dcedf204/users/b4ca49c8a2cf4575a326df65ef5d0bd4/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:36.617473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50137c-494c-41bb-9466-b027c556d906 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:36.618133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50137c-494c-41bb-9466-b027c556d906 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:36.618570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3b94b-e7f9-4a2e-8004-7bd777622235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:36.618751 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50137c-494c-41bb-9466-b027c556d906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:36.619298 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3b94b-e7f9-4a2e-8004-7bd777622235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:36.619615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3b94b-e7f9-4a2e-8004-7bd777622235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:36.619924 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3b94b-e7f9-4a2e-8004-7bd777622235 admin admin] PATH_INFO: `/v3/projects/a5eed99762724bc69ca69883429093fc` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:36.621678 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3b94b-e7f9-4a2e-8004-7bd777622235 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5eed99762724bc69ca69883429093fc)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:36.626302 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3b94b-e7f9-4a2e-8004-7bd777622235 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:36.711764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-de50137c-494c-41bb-9466-b027c556d906 None None] MFA Rules not processed for user `b4ca49c8a2cf4575a326df65ef5d0bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:36.715667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0fd3b94b-e7f9-4a2e-8004-7bd777622235 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5eed99762724bc69ca69883429093fc', 'request_id': 'req-0fd3b94b-e7f9-4a2e-8004-7bd777622235'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:36.718553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-de50137c-494c-41bb-9466-b027c556d906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:36.729842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0fd3b94b-e7f9-4a2e-8004-7bd777622235 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5eed99762724bc69ca69883429093fc', 'request_id': 'req-0fd3b94b-e7f9-4a2e-8004-7bd777622235'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:36.745522 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0fd3b94b-e7f9-4a2e-8004-7bd777622235 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5eed99762724bc69ca69883429093fc', 'request_id': 'req-0fd3b94b-e7f9-4a2e-8004-7bd777622235'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:36.781337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:36.790431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0fd3b94b-e7f9-4a2e-8004-7bd777622235 admin admin] The token cache is being invalidate because project a5eed99762724bc69ca69883429093fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:24:36.792512 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0fd3b94b-e7f9-4a2e-8004-7bd777622235 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:36.794504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0fd3b94b-e7f9-4a2e-8004-7bd777622235 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:36.807195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 835/1691] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:24:36 2022] DELETE /identity/v3/projects/a5eed99762724bc69ca69883429093fc => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:36.812919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409398a-9e44-4034-8ab1-777c9f385f61 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:36.814498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409398a-9e44-4034-8ab1-777c9f385f61 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:36.815546 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409398a-9e44-4034-8ab1-777c9f385f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:36.903066 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5409398a-9e44-4034-8ab1-777c9f385f61 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:36.906901 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5409398a-9e44-4034-8ab1-777c9f385f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:37.072102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 857/1692] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:36 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:37.094279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50137c-494c-41bb-9466-b027c556d906 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:37.098418 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-de50137c-494c-41bb-9466-b027c556d906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:37.172049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:37.226608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 836/1693] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:37.232510 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b45d34-3161-4b24-a27f-f165be051a9a None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:37.234688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b45d34-3161-4b24-a27f-f165be051a9a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:37.235627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b45d34-3161-4b24-a27f-f165be051a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:37.287378 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-50b45d34-3161-4b24-a27f-f165be051a9a None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:37.288848 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-50b45d34-3161-4b24-a27f-f165be051a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:37.342932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 837/1694] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:37.348996 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b45d34-3161-4b24-a27f-f165be051a9a None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:37.370075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b45d34-3161-4b24-a27f-f165be051a9a None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:37.380086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64e33d-584e-4edd-96d3-b26b9aebf735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:37.380865 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64e33d-584e-4edd-96d3-b26b9aebf735 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:37.382060 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64e33d-584e-4edd-96d3-b26b9aebf735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:37.383765 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64e33d-584e-4edd-96d3-b26b9aebf735 admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:37.385014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50137c-494c-41bb-9466-b027c556d906 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:37.385442 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64e33d-584e-4edd-96d3-b26b9aebf735 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:24:37.385922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64e33d-584e-4edd-96d3-b26b9aebf735 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:37.389761 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64e33d-584e-4edd-96d3-b26b9aebf735 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:37.422363 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63831c6f-176f-4d09-b3a7-30142638b584 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:37.423460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63831c6f-176f-4d09-b3a7-30142638b584 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:37.423956 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63831c6f-176f-4d09-b3a7-30142638b584 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:37.424472 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63831c6f-176f-4d09-b3a7-30142638b584 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:37.436536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 838/1695] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:24:37.446186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64e33d-584e-4edd-96d3-b26b9aebf735 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:37.457764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63831c6f-176f-4d09-b3a7-30142638b584 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:37.462786 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63831c6f-176f-4d09-b3a7-30142638b584 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:37.466048 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64e33d-584e-4edd-96d3-b26b9aebf735 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:37.481757 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76baca-127a-477e-a02b-6ded38a78c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:37.484045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76baca-127a-477e-a02b-6ded38a78c75 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:37.484045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76baca-127a-477e-a02b-6ded38a78c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:37.484644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76baca-127a-477e-a02b-6ded38a78c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:37.485486 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76baca-127a-477e-a02b-6ded38a78c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:37.491349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76baca-127a-477e-a02b-6ded38a78c75 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:37.504559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 858/1696] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:37 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:37.559092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 839/1697] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:37 2022] POST /identity/v3/projects => generated 363 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:37.580698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63831c6f-176f-4d09-b3a7-30142638b584 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:37.619362 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63831c6f-176f-4d09-b3a7-30142638b584 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:37.726392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76baca-127a-477e-a02b-6ded38a78c75 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:37.775777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da2acb-a383-435b-8169-e52c75d88a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:37.775777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da2acb-a383-435b-8169-e52c75d88a08 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:37.775777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da2acb-a383-435b-8169-e52c75d88a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:37.776382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da2acb-a383-435b-8169-e52c75d88a08 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:37.776843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da2acb-a383-435b-8169-e52c75d88a08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:37.788036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da2acb-a383-435b-8169-e52c75d88a08 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:37.862982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 859/1698] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:37 2022] POST /identity/v3/users => generated 282 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:37.865480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da2acb-a383-435b-8169-e52c75d88a08 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:37.892826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76baca-127a-477e-a02b-6ded38a78c75 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:37.907682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da2acb-a383-435b-8169-e52c75d88a08 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:37.917499 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a14e3-8b62-4687-9232-a913034640a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:37.919031 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a14e3-8b62-4687-9232-a913034640a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:37.920511 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a14e3-8b62-4687-9232-a913034640a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:37.920836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a14e3-8b62-4687-9232-a913034640a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:37.926441 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-1f7a14e3-8b62-4687-9232-a913034640a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:37.929205 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69f0db-fc44-4fa4-9ae8-756bc9078d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:37.930275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69f0db-fc44-4fa4-9ae8-756bc9078d2f admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:37.930805 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69f0db-fc44-4fa4-9ae8-756bc9078d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:37.931282 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69f0db-fc44-4fa4-9ae8-756bc9078d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:37.932494 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69f0db-fc44-4fa4-9ae8-756bc9078d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:37.936584 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69f0db-fc44-4fa4-9ae8-756bc9078d2f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:37.997675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 860/1699] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:38.000257 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69f0db-fc44-4fa4-9ae8-756bc9078d2f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:38.064104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69f0db-fc44-4fa4-9ae8-756bc9078d2f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:38.092854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e3d6f-6884-4f07-ba64-cff48187e828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:38.094027 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e3d6f-6884-4f07-ba64-cff48187e828 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:38.094502 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e3d6f-6884-4f07-ba64-cff48187e828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:38.095006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e3d6f-6884-4f07-ba64-cff48187e828 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:38.096202 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e3d6f-6884-4f07-ba64-cff48187e828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:38.099865 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e3d6f-6884-4f07-ba64-cff48187e828 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:38.164420 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 861/1700] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:38.167512 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e3d6f-6884-4f07-ba64-cff48187e828 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:38.191951 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e3d6f-6884-4f07-ba64-cff48187e828 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:38.204278 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85dc67-3400-4f20-a83f-827cc7cddba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:38.205867 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85dc67-3400-4f20-a83f-827cc7cddba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:38.206369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85dc67-3400-4f20-a83f-827cc7cddba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:38.206995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85dc67-3400-4f20-a83f-827cc7cddba2 admin admin] PATH_INFO: `/v3/projects/9e50c3667a884cc496473dd83535f68f/users/6a17d414979a43db96cf6cef885bbe9f/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:38.241810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85dc67-3400-4f20-a83f-827cc7cddba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e50c3667a884cc496473dd83535f68f, user_id=6a17d414979a43db96cf6cef885bbe9f, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:38.249753 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85dc67-3400-4f20-a83f-827cc7cddba2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:38.263164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:38.272161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7a14e3-8b62-4687-9232-a913034640a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:38.273865 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7a14e3-8b62-4687-9232-a913034640a6 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:38.314791 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 862/1701] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:38 2022] PUT /identity/v3/projects/9e50c3667a884cc496473dd83535f68f/users/6a17d414979a43db96cf6cef885bbe9f/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:38.331540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0328c65-727e-4c2b-93a3-cbedaf25cc9f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:38.332766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0328c65-727e-4c2b-93a3-cbedaf25cc9f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:38.334850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0328c65-727e-4c2b-93a3-cbedaf25cc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:38.356074 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 840/1702] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:38.419610 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a0328c65-727e-4c2b-93a3-cbedaf25cc9f None None] MFA Rules not processed for user `6a17d414979a43db96cf6cef885bbe9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:38.424153 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a0328c65-727e-4c2b-93a3-cbedaf25cc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:38.761992 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 863/1703] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:38.769907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a14e3-8b62-4687-9232-a913034640a6 service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:38.816526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a14e3-8b62-4687-9232-a913034640a6 service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:39.074282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2f6d9-48b6-43ac-80f7-40837981bb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:39.079005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2f6d9-48b6-43ac-80f7-40837981bb6a admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:39.079610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2f6d9-48b6-43ac-80f7-40837981bb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:39.079967 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2f6d9-48b6-43ac-80f7-40837981bb6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:39.082830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2f6d9-48b6-43ac-80f7-40837981bb6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:39.092859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2f6d9-48b6-43ac-80f7-40837981bb6a admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:39.179656 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:39.191575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 841/1704] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:38 2022] POST /identity/v3/projects => generated 365 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:39.197084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0328c65-727e-4c2b-93a3-cbedaf25cc9f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:39.223528 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0328c65-727e-4c2b-93a3-cbedaf25cc9f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:39.456271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30862082-3928-46d0-abc7-7661b24e4706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:39.457183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30862082-3928-46d0-abc7-7661b24e4706 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:39.457638 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30862082-3928-46d0-abc7-7661b24e4706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:39.458061 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30862082-3928-46d0-abc7-7661b24e4706 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:39.458860 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30862082-3928-46d0-abc7-7661b24e4706 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:39.461119 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30862082-3928-46d0-abc7-7661b24e4706 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:39.506497 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 864/1705] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:39 2022] POST /identity/v3/users => generated 289 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:39.508667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2f6d9-48b6-43ac-80f7-40837981bb6a admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:39.543778 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2f6d9-48b6-43ac-80f7-40837981bb6a admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:39.566620 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafe00c-3131-4c5f-a0da-c6ce734d72eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:39.567095 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafe00c-3131-4c5f-a0da-c6ce734d72eb admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:39.567323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafe00c-3131-4c5f-a0da-c6ce734d72eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:39.569409 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafe00c-3131-4c5f-a0da-c6ce734d72eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:39.570507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafe00c-3131-4c5f-a0da-c6ce734d72eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:39.589944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafe00c-3131-4c5f-a0da-c6ce734d72eb admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:39.632342 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 842/1706] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:39.639669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30862082-3928-46d0-abc7-7661b24e4706 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:39.704143 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30862082-3928-46d0-abc7-7661b24e4706 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:39.746365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad4ea4-edbc-494c-83df-d647cdf17461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:39.747425 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad4ea4-edbc-494c-83df-d647cdf17461 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:39.748580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad4ea4-edbc-494c-83df-d647cdf17461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:39.749031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad4ea4-edbc-494c-83df-d647cdf17461 admin admin] PATH_INFO: `/v3/projects/e2010f0ea84e4e1b8a421d5b40806d8d/users/a7891eeea5124ec684f7b95764046735/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:39.867167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad4ea4-edbc-494c-83df-d647cdf17461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2010f0ea84e4e1b8a421d5b40806d8d, user_id=a7891eeea5124ec684f7b95764046735, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:39.885937 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad4ea4-edbc-494c-83df-d647cdf17461 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:39.984259 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 865/1707] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:39 2022] PUT /identity/v3/projects/e2010f0ea84e4e1b8a421d5b40806d8d/users/a7891eeea5124ec684f7b95764046735/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:39.995649 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafe00c-3131-4c5f-a0da-c6ce734d72eb admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:40.025661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafe00c-3131-4c5f-a0da-c6ce734d72eb admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:40.224911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca811b08-aa9a-4b8b-8d71-a7c1e94e37ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:40.225969 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca811b08-aa9a-4b8b-8d71-a7c1e94e37ba admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:40.226373 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca811b08-aa9a-4b8b-8d71-a7c1e94e37ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:40.226653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca811b08-aa9a-4b8b-8d71-a7c1e94e37ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:40.227348 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca811b08-aa9a-4b8b-8d71-a7c1e94e37ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:40.229594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca811b08-aa9a-4b8b-8d71-a7c1e94e37ba admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:40.275346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 843/1708] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:24:40.281390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad4ea4-edbc-494c-83df-d647cdf17461 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:40.320402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad4ea4-edbc-494c-83df-d647cdf17461 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:40.350522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904d01a-3495-47a8-8c19-d2786fc63d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:40.351243 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904d01a-3495-47a8-8c19-d2786fc63d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:40.351530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904d01a-3495-47a8-8c19-d2786fc63d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:40.351873 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904d01a-3495-47a8-8c19-d2786fc63d56 admin admin] PATH_INFO: `/v3/projects/e2010f0ea84e4e1b8a421d5b40806d8d/users/a7891eeea5124ec684f7b95764046735/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:40.370267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904d01a-3495-47a8-8c19-d2786fc63d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2010f0ea84e4e1b8a421d5b40806d8d, user_id=a7891eeea5124ec684f7b95764046735, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:40.370267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904d01a-3495-47a8-8c19-d2786fc63d56 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:40.409316 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 866/1709] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:40 2022] PUT /identity/v3/projects/e2010f0ea84e4e1b8a421d5b40806d8d/users/a7891eeea5124ec684f7b95764046735/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:40.429604 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28943f75-8d79-47ff-86d1-9fb9c90ca8c3 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:40.429938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28943f75-8d79-47ff-86d1-9fb9c90ca8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:40.431646 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28943f75-8d79-47ff-86d1-9fb9c90ca8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:40.524695 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b54b8-04c6-45d7-a59d-c9847fe9037a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:40.527417 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b54b8-04c6-45d7-a59d-c9847fe9037a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:40.529351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b54b8-04c6-45d7-a59d-c9847fe9037a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:40.567848 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-28943f75-8d79-47ff-86d1-9fb9c90ca8c3 None None] MFA Rules not processed for user `a7891eeea5124ec684f7b95764046735`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:40.578080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-28943f75-8d79-47ff-86d1-9fb9c90ca8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:40.631640 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-4e0b54b8-04c6-45d7-a59d-c9847fe9037a None None] MFA Rules not processed for user `1db1a9a77be6452991ec4f99af814e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:40.633329 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4e0b54b8-04c6-45d7-a59d-c9847fe9037a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:40.795564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:40.880802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 844/1710] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:24:40.886661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28943f75-8d79-47ff-86d1-9fb9c90ca8c3 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:40.933674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 867/1711] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:40.939978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc93d2-d80b-4222-bd06-79aa65281d4d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:40.940282 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc93d2-d80b-4222-bd06-79aa65281d4d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:40.940570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc93d2-d80b-4222-bd06-79aa65281d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:41.013175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-48fc93d2-d80b-4222-bd06-79aa65281d4d None None] MFA Rules not processed for user `e720fd25d54b4dc6b3387ea762d7e4ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:41.020287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-48fc93d2-d80b-4222-bd06-79aa65281d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:41.061235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28943f75-8d79-47ff-86d1-9fb9c90ca8c3 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:41.073611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bef8f0-a71c-4d7e-9002-0a92389e32eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:41.074507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bef8f0-a71c-4d7e-9002-0a92389e32eb service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:41.075007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bef8f0-a71c-4d7e-9002-0a92389e32eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:41.075521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bef8f0-a71c-4d7e-9002-0a92389e32eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:41.102300 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bef8f0-a71c-4d7e-9002-0a92389e32eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:41.105127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bef8f0-a71c-4d7e-9002-0a92389e32eb service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:41.172026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 845/1712] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:40 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:41.173880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bef8f0-a71c-4d7e-9002-0a92389e32eb service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:41.176093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e7bef8f0-a71c-4d7e-9002-0a92389e32eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:41.210009 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:41.354735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 868/1713] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:41.386150 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc93d2-d80b-4222-bd06-79aa65281d4d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:41.389528 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-48fc93d2-d80b-4222-bd06-79aa65281d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:41.429834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bef8f0-a71c-4d7e-9002-0a92389e32eb service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:41.453317 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51821bfb-0695-46c2-8ccb-391fefda026c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:41.454344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51821bfb-0695-46c2-8ccb-391fefda026c service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:41.454811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51821bfb-0695-46c2-8ccb-391fefda026c service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:41.455278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51821bfb-0695-46c2-8ccb-391fefda026c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:41.467339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc93d2-d80b-4222-bd06-79aa65281d4d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:41.488370 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51821bfb-0695-46c2-8ccb-391fefda026c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:41.491560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51821bfb-0695-46c2-8ccb-391fefda026c service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:41.502911 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cc298-057f-4b69-8a67-90d23321b102 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:41.503560 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cc298-057f-4b69-8a67-90d23321b102 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:41.503863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cc298-057f-4b69-8a67-90d23321b102 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:41.504438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cc298-057f-4b69-8a67-90d23321b102 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:41.526805 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 846/1714] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:41.570168 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cc298-057f-4b69-8a67-90d23321b102 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:41.571950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cc298-057f-4b69-8a67-90d23321b102 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:41.645715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 869/1715] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:24:43.254197 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51821bfb-0695-46c2-8ccb-391fefda026c service nova] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:43.256808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-51821bfb-0695-46c2-8ccb-391fefda026c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:43.297224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51821bfb-0695-46c2-8ccb-391fefda026c service nova] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:43.324965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b4613-737b-47ef-adae-7c88dc39dd10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:43.359126 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b4613-737b-47ef-adae-7c88dc39dd10 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:43.359772 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b4613-737b-47ef-adae-7c88dc39dd10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:43.362314 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b4613-737b-47ef-adae-7c88dc39dd10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:43.378772 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-572b4613-737b-47ef-adae-7c88dc39dd10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:43.828076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b4613-737b-47ef-adae-7c88dc39dd10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:43.838599 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b4613-737b-47ef-adae-7c88dc39dd10 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:43.900568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 847/1716] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:44.074970 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cc298-057f-4b69-8a67-90d23321b102 service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:44.293540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cc298-057f-4b69-8a67-90d23321b102 service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:44.313166 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec7e2a-70e6-405b-976f-998c88e36eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:44.314295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec7e2a-70e6-405b-976f-998c88e36eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:44.314804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec7e2a-70e6-405b-976f-998c88e36eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:44.315303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec7e2a-70e6-405b-976f-998c88e36eb3 admin admin] PATH_INFO: `/v3/users/6a17d414979a43db96cf6cef885bbe9f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:44.322620 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec7e2a-70e6-405b-976f-998c88e36eb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a17d414979a43db96cf6cef885bbe9f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:44.326692 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec7e2a-70e6-405b-976f-998c88e36eb3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:44.560950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:44.564371 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-61ec7e2a-70e6-405b-976f-998c88e36eb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a17d414979a43db96cf6cef885bbe9f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:44.626840 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-61ec7e2a-70e6-405b-976f-998c88e36eb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a17d414979a43db96cf6cef885bbe9f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:44.652162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-61ec7e2a-70e6-405b-976f-998c88e36eb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a17d414979a43db96cf6cef885bbe9f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:44.656287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-61ec7e2a-70e6-405b-976f-998c88e36eb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a17d414979a43db96cf6cef885bbe9f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:44.691532 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-61ec7e2a-70e6-405b-976f-998c88e36eb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a17d414979a43db96cf6cef885bbe9f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:44.707803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 870/1717] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:24:44 2022] DELETE /identity/v3/users/6a17d414979a43db96cf6cef885bbe9f => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:44.713731 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7584c4-e3b8-4dd4-9b94-bc409a73694c None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:44.714813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7584c4-e3b8-4dd4-9b94-bc409a73694c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:44.715323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7584c4-e3b8-4dd4-9b94-bc409a73694c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:44.772016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-1d7584c4-e3b8-4dd4-9b94-bc409a73694c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:44.778207 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-1d7584c4-e3b8-4dd4-9b94-bc409a73694c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:45.020184 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 848/1718] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:45.172449 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec7e2a-70e6-405b-976f-998c88e36eb3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:45.175822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-61ec7e2a-70e6-405b-976f-998c88e36eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:45.593639 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec7e2a-70e6-405b-976f-998c88e36eb3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:45.611487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a1f82-77ec-4282-8308-d9624e4aa446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:45.612311 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a1f82-77ec-4282-8308-d9624e4aa446 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:45.612991 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a1f82-77ec-4282-8308-d9624e4aa446 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:45.614172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a1f82-77ec-4282-8308-d9624e4aa446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:45.647963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a1f82-77ec-4282-8308-d9624e4aa446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:45.655415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a1f82-77ec-4282-8308-d9624e4aa446 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:45.690893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 871/1719] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:47.024206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f235b-daef-4668-8361-6b1d07ac92e7 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:47.025756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f235b-daef-4668-8361-6b1d07ac92e7 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:47.027366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f235b-daef-4668-8361-6b1d07ac92e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:47.219843 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-e64f235b-daef-4668-8361-6b1d07ac92e7 None None] MFA Rules not processed for user `78b05a538ce0466ea8b3e27a0a6fc162`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:47.222449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e64f235b-daef-4668-8361-6b1d07ac92e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:47.406537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:47.528089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a366dd3-57b2-410a-8587-9df76a59f820 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:47.528708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a366dd3-57b2-410a-8587-9df76a59f820 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:47.528708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a366dd3-57b2-410a-8587-9df76a59f820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:47.600659 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-3a366dd3-57b2-410a-8587-9df76a59f820 None None] MFA Rules not processed for user `c06cd43d533f4621b0a69dbb0ef22dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:47.608265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3a366dd3-57b2-410a-8587-9df76a59f820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:47.621673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 849/1720] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 599 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:47.631335 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8bd05-3101-4a64-979a-4c2af802c3ef None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:47.631919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8bd05-3101-4a64-979a-4c2af802c3ef None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:47.632371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8bd05-3101-4a64-979a-4c2af802c3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:47.746402 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-2eb8bd05-3101-4a64-979a-4c2af802c3ef None None] MFA Rules not processed for user `78b05a538ce0466ea8b3e27a0a6fc162`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:47.754482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-2eb8bd05-3101-4a64-979a-4c2af802c3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:47.809678 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 850/1721] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:47.822649 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8bd05-3101-4a64-979a-4c2af802c3ef None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:47.826636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-2eb8bd05-3101-4a64-979a-4c2af802c3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:47.875834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 872/1722] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:24:48.415119 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8bd05-3101-4a64-979a-4c2af802c3ef None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:48.465896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848e4fa-f0b9-4bab-aad7-46d3d9580d25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:48.472405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848e4fa-f0b9-4bab-aad7-46d3d9580d25 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:48.473390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848e4fa-f0b9-4bab-aad7-46d3d9580d25 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:48.475166 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848e4fa-f0b9-4bab-aad7-46d3d9580d25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:48.514708 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848e4fa-f0b9-4bab-aad7-46d3d9580d25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:48.518434 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848e4fa-f0b9-4bab-aad7-46d3d9580d25 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:48.566335 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 851/1723] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 744 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:24:49.074842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f887f-0fb1-4107-bece-f4af4521ff12 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:49.076465 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f887f-0fb1-4107-bece-f4af4521ff12 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:49.076708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f887f-0fb1-4107-bece-f4af4521ff12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:49.163116 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848e4fa-f0b9-4bab-aad7-46d3d9580d25 service nova] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:49.164847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0848e4fa-f0b9-4bab-aad7-46d3d9580d25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:49.173752 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-146f887f-0fb1-4107-bece-f4af4521ff12 None None] MFA Rules not processed for user `e720fd25d54b4dc6b3387ea762d7e4ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:49.178142 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-146f887f-0fb1-4107-bece-f4af4521ff12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:49.193299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848e4fa-f0b9-4bab-aad7-46d3d9580d25 service nova] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:49.207715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812836d-c1eb-4fc0-a3fe-345be2e102a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:49.208340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812836d-c1eb-4fc0-a3fe-345be2e102a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:49.208804 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812836d-c1eb-4fc0-a3fe-345be2e102a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:49.209303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812836d-c1eb-4fc0-a3fe-345be2e102a9 admin admin] PATH_INFO: `/v3/users/1cdc655825d74cdbbf708032ad551ec7` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:49.213273 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812836d-c1eb-4fc0-a3fe-345be2e102a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cdc655825d74cdbbf708032ad551ec7)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:49.216360 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812836d-c1eb-4fc0-a3fe-345be2e102a9 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:49.344588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-a812836d-c1eb-4fc0-a3fe-345be2e102a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cdc655825d74cdbbf708032ad551ec7'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:49.346613 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-a812836d-c1eb-4fc0-a3fe-345be2e102a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cdc655825d74cdbbf708032ad551ec7'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:49.382431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-a812836d-c1eb-4fc0-a3fe-345be2e102a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cdc655825d74cdbbf708032ad551ec7'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:49.427702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-a812836d-c1eb-4fc0-a3fe-345be2e102a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cdc655825d74cdbbf708032ad551ec7'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:49.449883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-a812836d-c1eb-4fc0-a3fe-345be2e102a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cdc655825d74cdbbf708032ad551ec7'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:49.465790 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 852/1724] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:24:49 2022] DELETE /identity/v3/users/1cdc655825d74cdbbf708032ad551ec7 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:49.467966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812836d-c1eb-4fc0-a3fe-345be2e102a9 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:49.469723 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a812836d-c1eb-4fc0-a3fe-345be2e102a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:49.522448 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:49.522930 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 873/1725] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:49.533737 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f333b8e-4900-49b5-8da3-3abd2b186b61 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:49.534507 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f333b8e-4900-49b5-8da3-3abd2b186b61 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:49.535208 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f333b8e-4900-49b5-8da3-3abd2b186b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:49.626121 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9f333b8e-4900-49b5-8da3-3abd2b186b61 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:49.644067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9f333b8e-4900-49b5-8da3-3abd2b186b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:49.690750 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:49.730337 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812836d-c1eb-4fc0-a3fe-345be2e102a9 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:49.746884 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfd175-c99b-41ba-883b-8727426825f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:49.749187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfd175-c99b-41ba-883b-8727426825f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:49.752450 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfd175-c99b-41ba-883b-8727426825f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:49.753309 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfd175-c99b-41ba-883b-8727426825f3 admin admin] PATH_INFO: `/v3/projects/9e50c3667a884cc496473dd83535f68f` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:49.772974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfd175-c99b-41ba-883b-8727426825f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e50c3667a884cc496473dd83535f68f)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:49.776539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfd175-c99b-41ba-883b-8727426825f3 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:49.994738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 874/1726] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:50.076274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f333b8e-4900-49b5-8da3-3abd2b186b61 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:50.080551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9f333b8e-4900-49b5-8da3-3abd2b186b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:50.097164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0acfd175-c99b-41ba-883b-8727426825f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e50c3667a884cc496473dd83535f68f', 'request_id': 'req-0acfd175-c99b-41ba-883b-8727426825f3'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:50.129112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0acfd175-c99b-41ba-883b-8727426825f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e50c3667a884cc496473dd83535f68f', 'request_id': 'req-0acfd175-c99b-41ba-883b-8727426825f3'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:50.141097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0acfd175-c99b-41ba-883b-8727426825f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e50c3667a884cc496473dd83535f68f', 'request_id': 'req-0acfd175-c99b-41ba-883b-8727426825f3'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:50.195505 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0acfd175-c99b-41ba-883b-8727426825f3 admin admin] The token cache is being invalidate because project 9e50c3667a884cc496473dd83535f68f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:24:50.196318 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0acfd175-c99b-41ba-883b-8727426825f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:50.202127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0acfd175-c99b-41ba-883b-8727426825f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:50.229957 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 853/1727] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:24:49 2022] DELETE /identity/v3/projects/9e50c3667a884cc496473dd83535f68f => generated 0 bytes in 759 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:50.235221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfd175-c99b-41ba-883b-8727426825f3 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:50.240585 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0acfd175-c99b-41ba-883b-8727426825f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:50.335749 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f333b8e-4900-49b5-8da3-3abd2b186b61 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:50.459042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71c860-d438-406f-ba8e-58b756d59f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:50.459837 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71c860-d438-406f-ba8e-58b756d59f96 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:50.460235 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71c860-d438-406f-ba8e-58b756d59f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:50.460618 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71c860-d438-406f-ba8e-58b756d59f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:50.466820 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ba71c860-d438-406f-ba8e-58b756d59f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:50.562093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfd175-c99b-41ba-883b-8727426825f3 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:50.588257 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b7010-fbee-4a5f-ac03-a62f5b65cfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:50.589787 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b7010-fbee-4a5f-ac03-a62f5b65cfa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:50.589981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b7010-fbee-4a5f-ac03-a62f5b65cfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:50.590436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b7010-fbee-4a5f-ac03-a62f5b65cfa4 admin admin] PATH_INFO: `/v3/users/a7891eeea5124ec684f7b95764046735` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:50.594402 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b7010-fbee-4a5f-ac03-a62f5b65cfa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7891eeea5124ec684f7b95764046735)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:50.598464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b7010-fbee-4a5f-ac03-a62f5b65cfa4 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:50.745357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-3f6b7010-fbee-4a5f-ac03-a62f5b65cfa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7891eeea5124ec684f7b95764046735'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:50.748925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-3f6b7010-fbee-4a5f-ac03-a62f5b65cfa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7891eeea5124ec684f7b95764046735'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:50.752069 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:50.783614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71c860-d438-406f-ba8e-58b756d59f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:50.786265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71c860-d438-406f-ba8e-58b756d59f96 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:50.812664 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-3f6b7010-fbee-4a5f-ac03-a62f5b65cfa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7891eeea5124ec684f7b95764046735'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:50.845043 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-3f6b7010-fbee-4a5f-ac03-a62f5b65cfa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7891eeea5124ec684f7b95764046735'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:50.849884 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 875/1728] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 775 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:24:50.852474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdacea5-ccfb-4d23-aa08-580663099973 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:50.852780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdacea5-ccfb-4d23-aa08-580663099973 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:50.853315 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdacea5-ccfb-4d23-aa08-580663099973 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:50.855911 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 876/1729] 10.222.0.88 () {54 vars in 912 bytes} [Tue Aug 23 22:24:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:50.875373 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-3f6b7010-fbee-4a5f-ac03-a62f5b65cfa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7891eeea5124ec684f7b95764046735'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:50.884983 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d483b-81b7-4242-96fd-c150c12ef84e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:50.885377 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d483b-81b7-4242-96fd-c150c12ef84e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:50.885826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d483b-81b7-4242-96fd-c150c12ef84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:50.901606 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 854/1730] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:24:50 2022] DELETE /identity/v3/users/a7891eeea5124ec684f7b95764046735 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:51.061033 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9d4d483b-81b7-4242-96fd-c150c12ef84e None None] MFA Rules not processed for user `6505100494d4496591407c48b48535ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:51.063555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9d4d483b-81b7-4242-96fd-c150c12ef84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:51.080625 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:51.224086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e646f-79b3-428e-92f7-fd31a7fb8bf9 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:51.224949 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e646f-79b3-428e-92f7-fd31a7fb8bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:51.225417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e646f-79b3-428e-92f7-fd31a7fb8bf9 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:51.227870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 855/1731] 10.222.0.88 () {54 vars in 912 bytes} [Tue Aug 23 22:24:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:24:51.240160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397d7a9-1fe4-41ce-975f-6fa82770d9d5 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:51.240954 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397d7a9-1fe4-41ce-975f-6fa82770d9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:51.241401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397d7a9-1fe4-41ce-975f-6fa82770d9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:51.339774 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-7397d7a9-1fe4-41ce-975f-6fa82770d9d5 None None] MFA Rules not processed for user `6505100494d4496591407c48b48535ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:51.344575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-7397d7a9-1fe4-41ce-975f-6fa82770d9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:51.392906 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 877/1732] 10.222.0.88 () {58 vars in 1009 bytes} [Tue Aug 23 22:24:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 515 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:51.424135 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf652d0-d561-46c6-81a7-e76bccda24f9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:51.424620 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf652d0-d561-46c6-81a7-e76bccda24f9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:51.425110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf652d0-d561-46c6-81a7-e76bccda24f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:51.488672 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-5cf652d0-d561-46c6-81a7-e76bccda24f9 None None] MFA Rules not processed for user `1db1a9a77be6452991ec4f99af814e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:51.492260 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5cf652d0-d561-46c6-81a7-e76bccda24f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:51.598796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 856/1733] 10.222.0.88 () {58 vars in 1009 bytes} [Tue Aug 23 22:24:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:51.602422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397d7a9-1fe4-41ce-975f-6fa82770d9d5 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:51.632247 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-7397d7a9-1fe4-41ce-975f-6fa82770d9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:51.678491 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397d7a9-1fe4-41ce-975f-6fa82770d9d5 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:51.701904 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 878/1734] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:51.703309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf652d0-d561-46c6-81a7-e76bccda24f9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:51.705034 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3fa3a1-afbb-42db-b4a5-5e9128d7bc6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:51.706611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3fa3a1-afbb-42db-b4a5-5e9128d7bc6c service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:51.707346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3fa3a1-afbb-42db-b4a5-5e9128d7bc6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:51.707904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3fa3a1-afbb-42db-b4a5-5e9128d7bc6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:51.710511 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5cf652d0-d561-46c6-81a7-e76bccda24f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:51.725212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3fa3a1-afbb-42db-b4a5-5e9128d7bc6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:51.727446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3fa3a1-afbb-42db-b4a5-5e9128d7bc6c service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:51.747645 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 857/1735] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:51.750800 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af1a88-3631-4bb0-ab84-25f4aba53c4b None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:51.751476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af1a88-3631-4bb0-ab84-25f4aba53c4b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:51.752030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af1a88-3631-4bb0-ab84-25f4aba53c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:51.778599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf652d0-d561-46c6-81a7-e76bccda24f9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:51.802092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297aaab8-3c20-46f7-b0aa-e7e4c77c5e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:51.802092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297aaab8-3c20-46f7-b0aa-e7e4c77c5e81 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:51.802908 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297aaab8-3c20-46f7-b0aa-e7e4c77c5e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:51.802908 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297aaab8-3c20-46f7-b0aa-e7e4c77c5e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:51.809593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-85af1a88-3631-4bb0-ab84-25f4aba53c4b None None] MFA Rules not processed for user `4e884c4f24db45a786618fc28d19bced`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:51.812116 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-85af1a88-3631-4bb0-ab84-25f4aba53c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:51.830164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297aaab8-3c20-46f7-b0aa-e7e4c77c5e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:51.831976 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297aaab8-3c20-46f7-b0aa-e7e4c77c5e81 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:51.864836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 879/1736] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:52.101302 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 858/1737] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:52.108570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2898c7c-9bbc-4dd1-88f6-1eb5094544de None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:52.109133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2898c7c-9bbc-4dd1-88f6-1eb5094544de None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:52.109133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2898c7c-9bbc-4dd1-88f6-1eb5094544de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:52.141516 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a2898c7c-9bbc-4dd1-88f6-1eb5094544de None None] MFA Rules not processed for user `4e884c4f24db45a786618fc28d19bced`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:52.142882 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a2898c7c-9bbc-4dd1-88f6-1eb5094544de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:52.156487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 880/1738] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:53.289449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af1a88-3631-4bb0-ab84-25f4aba53c4b None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:53.293886 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-85af1a88-3631-4bb0-ab84-25f4aba53c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:53.402031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2898c7c-9bbc-4dd1-88f6-1eb5094544de None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:53.405802 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a2898c7c-9bbc-4dd1-88f6-1eb5094544de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:53.684036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:53.721355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af1a88-3631-4bb0-ab84-25f4aba53c4b None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:53.734340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5daa98-0767-4a9d-81f5-724946c35be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:53.735956 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5daa98-0767-4a9d-81f5-724946c35be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:53.737135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5daa98-0767-4a9d-81f5-724946c35be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:53.737840 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5daa98-0767-4a9d-81f5-724946c35be9 admin admin] PATH_INFO: `/v3/projects/dc6d5bd146c84060bb02700eddd537a2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:53.740989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5daa98-0767-4a9d-81f5-724946c35be9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc6d5bd146c84060bb02700eddd537a2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:53.745203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5daa98-0767-4a9d-81f5-724946c35be9 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:53.777529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2898c7c-9bbc-4dd1-88f6-1eb5094544de None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:53.790524 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b8c53-7683-4ba5-a143-c00917e0fa18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:53.802291 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b8c53-7683-4ba5-a143-c00917e0fa18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:53.802821 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b8c53-7683-4ba5-a143-c00917e0fa18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:53.814561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b8c53-7683-4ba5-a143-c00917e0fa18 admin admin] PATH_INFO: `/v3/projects/e2010f0ea84e4e1b8a421d5b40806d8d` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:53.816876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b8c53-7683-4ba5-a143-c00917e0fa18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2010f0ea84e4e1b8a421d5b40806d8d)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:53.825016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b8c53-7683-4ba5-a143-c00917e0fa18 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:53.859005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f5daa98-0767-4a9d-81f5-724946c35be9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc6d5bd146c84060bb02700eddd537a2', 'request_id': 'req-4f5daa98-0767-4a9d-81f5-724946c35be9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:53.882812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f5daa98-0767-4a9d-81f5-724946c35be9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc6d5bd146c84060bb02700eddd537a2', 'request_id': 'req-4f5daa98-0767-4a9d-81f5-724946c35be9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:53.896333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f5daa98-0767-4a9d-81f5-724946c35be9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc6d5bd146c84060bb02700eddd537a2', 'request_id': 'req-4f5daa98-0767-4a9d-81f5-724946c35be9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:53.924447 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f5daa98-0767-4a9d-81f5-724946c35be9 admin admin] The token cache is being invalidate because project dc6d5bd146c84060bb02700eddd537a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:24:53.924720 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f5daa98-0767-4a9d-81f5-724946c35be9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:53.925814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f5daa98-0767-4a9d-81f5-724946c35be9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:53.932934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 859/1739] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:24:53 2022] DELETE /identity/v3/projects/dc6d5bd146c84060bb02700eddd537a2 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:53.932934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8260a-fde8-4c6b-bd6c-802f95d8782d None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:53.932934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8260a-fde8-4c6b-bd6c-802f95d8782d None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:53.932934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8260a-fde8-4c6b-bd6c-802f95d8782d None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:53.943828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 860/1740] 10.222.0.88 () {54 vars in 961 bytes} [Tue Aug 23 22:24:53 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:53.945328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8260a-fde8-4c6b-bd6c-802f95d8782d None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:53.961095 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-83c8260a-fde8-4c6b-bd6c-802f95d8782d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:53.979920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:53.994561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-368b8c53-7683-4ba5-a143-c00917e0fa18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2010f0ea84e4e1b8a421d5b40806d8d', 'request_id': 'req-368b8c53-7683-4ba5-a143-c00917e0fa18'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:54.012445 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-368b8c53-7683-4ba5-a143-c00917e0fa18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2010f0ea84e4e1b8a421d5b40806d8d', 'request_id': 'req-368b8c53-7683-4ba5-a143-c00917e0fa18'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:54.041269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-368b8c53-7683-4ba5-a143-c00917e0fa18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2010f0ea84e4e1b8a421d5b40806d8d', 'request_id': 'req-368b8c53-7683-4ba5-a143-c00917e0fa18'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:54.068353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-368b8c53-7683-4ba5-a143-c00917e0fa18 admin admin] The token cache is being invalidate because project e2010f0ea84e4e1b8a421d5b40806d8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:24:54.068850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-368b8c53-7683-4ba5-a143-c00917e0fa18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:54.070508 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-368b8c53-7683-4ba5-a143-c00917e0fa18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:54.072367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 881/1741] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:24:53 2022] DELETE /identity/v3/projects/e2010f0ea84e4e1b8a421d5b40806d8d => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:54.073533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b8c53-7683-4ba5-a143-c00917e0fa18 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:54.075608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-368b8c53-7683-4ba5-a143-c00917e0fa18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:54.139852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8260a-fde8-4c6b-bd6c-802f95d8782d None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:54.146415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307a0da-b949-4443-9889-85b8eb4037bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e642dd7a07ca4afaa40c2769299d8bad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:54.147261 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307a0da-b949-4443-9889-85b8eb4037bf service placement] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:54.147648 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307a0da-b949-4443-9889-85b8eb4037bf service placement] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:54.147968 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307a0da-b949-4443-9889-85b8eb4037bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:54.150838 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a307a0da-b949-4443-9889-85b8eb4037bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:54.179588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a307a0da-b949-4443-9889-85b8eb4037bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:54.182892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a307a0da-b949-4443-9889-85b8eb4037bf service placement] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:54.198991 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 861/1742] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:54.200610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929891ad-a07e-4beb-892f-0171ab63b144 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:54.201029 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929891ad-a07e-4beb-892f-0171ab63b144 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:54.201592 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929891ad-a07e-4beb-892f-0171ab63b144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:54.252351 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-929891ad-a07e-4beb-892f-0171ab63b144 None None] MFA Rules not processed for user `e642dd7a07ca4afaa40c2769299d8bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:54.256019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-929891ad-a07e-4beb-892f-0171ab63b144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:54.281564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 862/1743] 10.222.0.88 () {58 vars in 1058 bytes} [Tue Aug 23 22:24:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:54.284058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e747cda-1f25-4c68-a290-f037e7e61c57 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:54.284529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e747cda-1f25-4c68-a290-f037e7e61c57 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:54.284991 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e747cda-1f25-4c68-a290-f037e7e61c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:54.333717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-4e747cda-1f25-4c68-a290-f037e7e61c57 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:54.335540 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4e747cda-1f25-4c68-a290-f037e7e61c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:54.351454 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 863/1744] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:54.352768 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e747cda-1f25-4c68-a290-f037e7e61c57 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:54.370026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b8c53-7683-4ba5-a143-c00917e0fa18 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:54.370431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e747cda-1f25-4c68-a290-f037e7e61c57 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:54.381715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36241e-1141-40cd-bf8e-7b13093c5d23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e642dd7a07ca4afaa40c2769299d8bad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:54.382787 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36241e-1141-40cd-bf8e-7b13093c5d23 service placement] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:54.383265 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36241e-1141-40cd-bf8e-7b13093c5d23 service placement] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:54.383873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36241e-1141-40cd-bf8e-7b13093c5d23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:54.393846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cad8dc-25ab-4fec-9c59-59a7368bc271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:54.394704 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cad8dc-25ab-4fec-9c59-59a7368bc271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:54.395275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cad8dc-25ab-4fec-9c59-59a7368bc271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:54.395557 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cad8dc-25ab-4fec-9c59-59a7368bc271 admin admin] PATH_INFO: `/v3/users/25c4cbb30c58482690a388192ba8b8a7` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:54.400099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36241e-1141-40cd-bf8e-7b13093c5d23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:54.402344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36241e-1141-40cd-bf8e-7b13093c5d23 service placement] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:54.402513 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cad8dc-25ab-4fec-9c59-59a7368bc271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25c4cbb30c58482690a388192ba8b8a7)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:54.405337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cad8dc-25ab-4fec-9c59-59a7368bc271 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:54.428361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 864/1745] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:54.431430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329555e5-483c-4109-abf0-ee77c20494ed None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:54.434344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329555e5-483c-4109-abf0-ee77c20494ed None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:54.435332 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329555e5-483c-4109-abf0-ee77c20494ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:54.499050 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-329555e5-483c-4109-abf0-ee77c20494ed None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:54.501849 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-329555e5-483c-4109-abf0-ee77c20494ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:54.515439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 865/1746] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:54.524559 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329555e5-483c-4109-abf0-ee77c20494ed None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:54.554661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329555e5-483c-4109-abf0-ee77c20494ed None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:54.565782 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f674547-fe6e-434e-b118-83ec473293cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:54.568048 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f674547-fe6e-434e-b118-83ec473293cc admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:54.568715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f674547-fe6e-434e-b118-83ec473293cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:54.570762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f674547-fe6e-434e-b118-83ec473293cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:54.572416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f674547-fe6e-434e-b118-83ec473293cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:24:54.573312 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f674547-fe6e-434e-b118-83ec473293cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:54.577919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f674547-fe6e-434e-b118-83ec473293cc admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:54.580046 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-41cad8dc-25ab-4fec-9c59-59a7368bc271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25c4cbb30c58482690a388192ba8b8a7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:54.604492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 866/1747] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:24:54.610846 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f674547-fe6e-434e-b118-83ec473293cc admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:54.619376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-41cad8dc-25ab-4fec-9c59-59a7368bc271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25c4cbb30c58482690a388192ba8b8a7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:54.634437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-41cad8dc-25ab-4fec-9c59-59a7368bc271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25c4cbb30c58482690a388192ba8b8a7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:54.635149 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-41cad8dc-25ab-4fec-9c59-59a7368bc271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25c4cbb30c58482690a388192ba8b8a7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:54.654742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-41cad8dc-25ab-4fec-9c59-59a7368bc271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25c4cbb30c58482690a388192ba8b8a7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:54.673279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 882/1748] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:24:54 2022] DELETE /identity/v3/users/25c4cbb30c58482690a388192ba8b8a7 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:54.694291 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f674547-fe6e-434e-b118-83ec473293cc admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:54.839772 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:55.041336 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8bded-0dd2-418f-9b8b-bb9513a678d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:55.042465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8bded-0dd2-418f-9b8b-bb9513a678d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:55.043060 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8bded-0dd2-418f-9b8b-bb9513a678d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:55.053390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8bded-0dd2-418f-9b8b-bb9513a678d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:55.057775 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8bded-0dd2-418f-9b8b-bb9513a678d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:55.070885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8bded-0dd2-418f-9b8b-bb9513a678d5 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:55.223258 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 867/1749] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:54 2022] POST /identity/v3/projects => generated 381 bytes in 613 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:55.227094 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cad8dc-25ab-4fec-9c59-59a7368bc271 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:55.241670 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-41cad8dc-25ab-4fec-9c59-59a7368bc271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:55.311730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cad8dc-25ab-4fec-9c59-59a7368bc271 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:55.523515 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:55.590196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc3fee-a970-403d-987d-e45d714fa068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:55.600894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc3fee-a970-403d-987d-e45d714fa068 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:55.602459 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc3fee-a970-403d-987d-e45d714fa068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:55.602820 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc3fee-a970-403d-987d-e45d714fa068 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:55.605344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc3fee-a970-403d-987d-e45d714fa068 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:55.615887 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc3fee-a970-403d-987d-e45d714fa068 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:55.689740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8bded-0dd2-418f-9b8b-bb9513a678d5 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:55.691789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ace8bded-0dd2-418f-9b8b-bb9513a678d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:55.790018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 883/1750] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:55 2022] POST /identity/v3/users => generated 291 bytes in 564 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:55.801267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc3fee-a970-403d-987d-e45d714fa068 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:55.882657 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc3fee-a970-403d-987d-e45d714fa068 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:55.919727 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7913053-d13d-4d07-9559-26d2138cbe7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:55.923654 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7913053-d13d-4d07-9559-26d2138cbe7c admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:55.931889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7913053-d13d-4d07-9559-26d2138cbe7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:55.933682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7913053-d13d-4d07-9559-26d2138cbe7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:55.938681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7913053-d13d-4d07-9559-26d2138cbe7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:55.959613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7913053-d13d-4d07-9559-26d2138cbe7c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:56.008248 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 884/1751] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:56.010338 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7913053-d13d-4d07-9559-26d2138cbe7c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:56.039665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8bded-0dd2-418f-9b8b-bb9513a678d5 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:56.066483 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02e690-821a-48c7-9b61-b6f3c1b68167 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:56.067331 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02e690-821a-48c7-9b61-b6f3c1b68167 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:56.067622 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02e690-821a-48c7-9b61-b6f3c1b68167 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:56.067893 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02e690-821a-48c7-9b61-b6f3c1b68167 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:56.081844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4e02e690-821a-48c7-9b61-b6f3c1b68167 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:56.109951 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7913053-d13d-4d07-9559-26d2138cbe7c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:56.157363 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb5a7b-953a-4e55-af02-e0c7b1387512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:56.158179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb5a7b-953a-4e55-af02-e0c7b1387512 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:56.166310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb5a7b-953a-4e55-af02-e0c7b1387512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:56.166765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb5a7b-953a-4e55-af02-e0c7b1387512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:56.170074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb5a7b-953a-4e55-af02-e0c7b1387512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:56.178942 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb5a7b-953a-4e55-af02-e0c7b1387512 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:56.253637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 885/1752] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:56.255891 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb5a7b-953a-4e55-af02-e0c7b1387512 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:56.298256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb5a7b-953a-4e55-af02-e0c7b1387512 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:56.350088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff436fd2-0c22-4868-8172-9d2ea13eadc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:56.351681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff436fd2-0c22-4868-8172-9d2ea13eadc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:56.358337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff436fd2-0c22-4868-8172-9d2ea13eadc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:56.360764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff436fd2-0c22-4868-8172-9d2ea13eadc7 admin admin] PATH_INFO: `/v3/projects/93a4338bace34535aac26747f4e8d69c/users/ccd386dcbd264ab19a1dd4cbb83d0f2e/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:56.410645 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff436fd2-0c22-4868-8172-9d2ea13eadc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a4338bace34535aac26747f4e8d69c, user_id=ccd386dcbd264ab19a1dd4cbb83d0f2e, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:56.426748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff436fd2-0c22-4868-8172-9d2ea13eadc7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:56.476010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02e690-821a-48c7-9b61-b6f3c1b68167 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:56.482987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02e690-821a-48c7-9b61-b6f3c1b68167 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:56.501609 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 886/1753] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:56 2022] PUT /identity/v3/projects/93a4338bace34535aac26747f4e8d69c/users/ccd386dcbd264ab19a1dd4cbb83d0f2e/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:56.516253 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bf337-b4c6-43c0-93c6-342ec6b61876 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:56.516915 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bf337-b4c6-43c0-93c6-342ec6b61876 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:56.518204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bf337-b4c6-43c0-93c6-342ec6b61876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:56.537969 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 868/1754] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 848 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:56.550828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff294a6b-41ad-4d16-9403-3031eb78a5c7 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:56.551210 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff294a6b-41ad-4d16-9403-3031eb78a5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:56.551491 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff294a6b-41ad-4d16-9403-3031eb78a5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:56.687148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-ff294a6b-41ad-4d16-9403-3031eb78a5c7 None None] MFA Rules not processed for user `78b05a538ce0466ea8b3e27a0a6fc162`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:56.688321 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ff294a6b-41ad-4d16-9403-3031eb78a5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:56.718346 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-971bf337-b4c6-43c0-93c6-342ec6b61876 None None] MFA Rules not processed for user `ccd386dcbd264ab19a1dd4cbb83d0f2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:56.724980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-971bf337-b4c6-43c0-93c6-342ec6b61876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:57.066784 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:57.211073 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 869/1755] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 661 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:24:57.217201 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff294a6b-41ad-4d16-9403-3031eb78a5c7 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:57.218597 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 887/1756] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 704 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:57.219874 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bf337-b4c6-43c0-93c6-342ec6b61876 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:57.225475 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ff294a6b-41ad-4d16-9403-3031eb78a5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:57.389950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bf337-b4c6-43c0-93c6-342ec6b61876 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:57.405740 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a25a7-9c74-455d-924d-16b6d5932b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:57.407694 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a25a7-9c74-455d-924d-16b6d5932b46 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:57.408020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a25a7-9c74-455d-924d-16b6d5932b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:57.410129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a25a7-9c74-455d-924d-16b6d5932b46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:57.412176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a25a7-9c74-455d-924d-16b6d5932b46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:57.418248 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a25a7-9c74-455d-924d-16b6d5932b46 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:57.583153 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 888/1757] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:24:57 2022] POST /identity/v3/projects => generated 381 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:57.587952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a25a7-9c74-455d-924d-16b6d5932b46 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:57.624570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff294a6b-41ad-4d16-9403-3031eb78a5c7 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:57.691334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:57.789935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40163e-5c12-4b28-a886-236a1d210a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:57.791553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40163e-5c12-4b28-a886-236a1d210a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:57.791874 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40163e-5c12-4b28-a886-236a1d210a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:57.800486 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40163e-5c12-4b28-a886-236a1d210a42 admin admin] PATH_INFO: `/v3/projects/97a176d63e3a40018d84dea69a60ec9d` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:57.804242 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e40163e-5c12-4b28-a886-236a1d210a42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97a176d63e3a40018d84dea69a60ec9d)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:57.813525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e40163e-5c12-4b28-a886-236a1d210a42 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:57.961192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1e40163e-5c12-4b28-a886-236a1d210a42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97a176d63e3a40018d84dea69a60ec9d', 'request_id': 'req-1e40163e-5c12-4b28-a886-236a1d210a42'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:57.976264 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1e40163e-5c12-4b28-a886-236a1d210a42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97a176d63e3a40018d84dea69a60ec9d', 'request_id': 'req-1e40163e-5c12-4b28-a886-236a1d210a42'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:58.030395 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a25a7-9c74-455d-924d-16b6d5932b46 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:58.034358 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1e40163e-5c12-4b28-a886-236a1d210a42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97a176d63e3a40018d84dea69a60ec9d', 'request_id': 'req-1e40163e-5c12-4b28-a886-236a1d210a42'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:58.093615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1e40163e-5c12-4b28-a886-236a1d210a42 admin admin] The token cache is being invalidate because project 97a176d63e3a40018d84dea69a60ec9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:24:58.093615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1e40163e-5c12-4b28-a886-236a1d210a42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:58.100163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1e40163e-5c12-4b28-a886-236a1d210a42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:58.109127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 870/1758] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:24:57 2022] DELETE /identity/v3/projects/97a176d63e3a40018d84dea69a60ec9d => generated 0 bytes in 897 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:58.113968 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40163e-5c12-4b28-a886-236a1d210a42 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:58.130420 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-1e40163e-5c12-4b28-a886-236a1d210a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:58.297182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:58.317508 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d81902-fccc-4f9f-aba7-372d06bd87fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:58.323479 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d81902-fccc-4f9f-aba7-372d06bd87fa admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:58.326131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d81902-fccc-4f9f-aba7-372d06bd87fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:58.326492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d81902-fccc-4f9f-aba7-372d06bd87fa admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:58.327292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d81902-fccc-4f9f-aba7-372d06bd87fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:58.338520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d81902-fccc-4f9f-aba7-372d06bd87fa admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:58.438733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 889/1759] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:24:57 2022] POST /identity/v3/users => generated 297 bytes in 851 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:24:58.450351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d81902-fccc-4f9f-aba7-372d06bd87fa admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:58.461760 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-59d81902-fccc-4f9f-aba7-372d06bd87fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:58.461996 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40163e-5c12-4b28-a886-236a1d210a42 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:58.502502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42535a-9984-49e0-9152-dac13c1fa3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:58.503645 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42535a-9984-49e0-9152-dac13c1fa3cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:58.504143 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42535a-9984-49e0-9152-dac13c1fa3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:58.504813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42535a-9984-49e0-9152-dac13c1fa3cf admin admin] PATH_INFO: `/v3/users/3420c8f7c8aa4daa8cbcdb4448157fb9` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:58.513028 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e42535a-9984-49e0-9152-dac13c1fa3cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3420c8f7c8aa4daa8cbcdb4448157fb9)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:58.526738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d81902-fccc-4f9f-aba7-372d06bd87fa admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:58.530681 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e42535a-9984-49e0-9152-dac13c1fa3cf admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:58.547201 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48213ef-d794-4ab9-931b-8c9dbeeb1dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:58.555897 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48213ef-d794-4ab9-931b-8c9dbeeb1dee admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:58.557895 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48213ef-d794-4ab9-931b-8c9dbeeb1dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:58.560530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48213ef-d794-4ab9-931b-8c9dbeeb1dee admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:58.563849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48213ef-d794-4ab9-931b-8c9dbeeb1dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:58.574383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48213ef-d794-4ab9-931b-8c9dbeeb1dee admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:58.654952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 890/1760] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:58.663973 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48213ef-d794-4ab9-931b-8c9dbeeb1dee admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:58.766578 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4e42535a-9984-49e0-9152-dac13c1fa3cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3420c8f7c8aa4daa8cbcdb4448157fb9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:58.767123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48213ef-d794-4ab9-931b-8c9dbeeb1dee admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:58.768391 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4e42535a-9984-49e0-9152-dac13c1fa3cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3420c8f7c8aa4daa8cbcdb4448157fb9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:58.790420 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513f5c3-7fb2-4ec4-8054-9c1aca261324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:58.793804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513f5c3-7fb2-4ec4-8054-9c1aca261324 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:58.794959 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513f5c3-7fb2-4ec4-8054-9c1aca261324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:58.796887 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513f5c3-7fb2-4ec4-8054-9c1aca261324 admin admin] PATH_INFO: `/v3/projects/a5f16a955c714c0681526ec837b834e7/users/c90932bb87394c93b81fb7299c26afde/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:58.804021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4e42535a-9984-49e0-9152-dac13c1fa3cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3420c8f7c8aa4daa8cbcdb4448157fb9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:58.846505 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513f5c3-7fb2-4ec4-8054-9c1aca261324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5f16a955c714c0681526ec837b834e7, user_id=c90932bb87394c93b81fb7299c26afde, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:58.850410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513f5c3-7fb2-4ec4-8054-9c1aca261324 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:58.863660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4e42535a-9984-49e0-9152-dac13c1fa3cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3420c8f7c8aa4daa8cbcdb4448157fb9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:58.881653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 891/1761] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:58 2022] PUT /identity/v3/projects/a5f16a955c714c0681526ec837b834e7/users/c90932bb87394c93b81fb7299c26afde/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:24:58.883898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513f5c3-7fb2-4ec4-8054-9c1aca261324 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:58.889771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4e42535a-9984-49e0-9152-dac13c1fa3cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3420c8f7c8aa4daa8cbcdb4448157fb9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:24:58.919262 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 871/1762] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:24:58 2022] DELETE /identity/v3/users/3420c8f7c8aa4daa8cbcdb4448157fb9 => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:24:59.080752 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513f5c3-7fb2-4ec4-8054-9c1aca261324 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:59.123539 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:59.240961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6392f-37f3-42e2-bb3a-59e84731f3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:59.243564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6392f-37f3-42e2-bb3a-59e84731f3be admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:59.245484 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6392f-37f3-42e2-bb3a-59e84731f3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:59.246059 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6392f-37f3-42e2-bb3a-59e84731f3be admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:59.249104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6392f-37f3-42e2-bb3a-59e84731f3be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:59.260694 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6392f-37f3-42e2-bb3a-59e84731f3be admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:59.303634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 892/1763] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:24:59.304689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42535a-9984-49e0-9152-dac13c1fa3cf admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:59.306970 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4e42535a-9984-49e0-9152-dac13c1fa3cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:59.362308 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42535a-9984-49e0-9152-dac13c1fa3cf admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:59.380114 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ace4c-5780-444e-a5ee-78f181e48926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:59.381330 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ace4c-5780-444e-a5ee-78f181e48926 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:59.381828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ace4c-5780-444e-a5ee-78f181e48926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:59.382311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ace4c-5780-444e-a5ee-78f181e48926 admin admin] PATH_INFO: `/v3/projects/a5f16a955c714c0681526ec837b834e7/users/c90932bb87394c93b81fb7299c26afde/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:59.395695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ace4c-5780-444e-a5ee-78f181e48926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5f16a955c714c0681526ec837b834e7, user_id=c90932bb87394c93b81fb7299c26afde, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:59.409822 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ace4c-5780-444e-a5ee-78f181e48926 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:59.442661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 872/1764] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:24:59 2022] PUT /identity/v3/projects/a5f16a955c714c0681526ec837b834e7/users/c90932bb87394c93b81fb7299c26afde/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:24:59.454891 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfedb14-07b0-4f8a-a59e-ce9dd35812e3 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:59.455203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfedb14-07b0-4f8a-a59e-ce9dd35812e3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:59.455588 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfedb14-07b0-4f8a-a59e-ce9dd35812e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:59.521830 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fdfedb14-07b0-4f8a-a59e-ce9dd35812e3 None None] MFA Rules not processed for user `c90932bb87394c93b81fb7299c26afde`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:59.523127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fdfedb14-07b0-4f8a-a59e-ce9dd35812e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:59.730591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 893/1765] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:24:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:59.767127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ace4c-5780-444e-a5ee-78f181e48926 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:59.769951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d31ace4c-5780-444e-a5ee-78f181e48926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:00.092233 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ace4c-5780-444e-a5ee-78f181e48926 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:00.124797 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663c185-ad5b-4625-90e9-538cb2ae0ca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:00.126451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663c185-ad5b-4625-90e9-538cb2ae0ca2 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:00.127281 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663c185-ad5b-4625-90e9-538cb2ae0ca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:00.128150 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663c185-ad5b-4625-90e9-538cb2ae0ca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:00.160893 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663c185-ad5b-4625-90e9-538cb2ae0ca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:00.179563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663c185-ad5b-4625-90e9-538cb2ae0ca2 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:00.226079 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 873/1766] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:00.935370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfedb14-07b0-4f8a-a59e-ce9dd35812e3 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:00.938844 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fdfedb14-07b0-4f8a-a59e-ce9dd35812e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:01.378576 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfedb14-07b0-4f8a-a59e-ce9dd35812e3 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:01.407437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73764f3-8caf-48e5-9d71-778464be0437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:01.411270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73764f3-8caf-48e5-9d71-778464be0437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:01.411533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73764f3-8caf-48e5-9d71-778464be0437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:01.415688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73764f3-8caf-48e5-9d71-778464be0437 admin admin] PATH_INFO: `/v3/projects/53ef6b68509e4cc8a40442cf6c158f75` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:01.423359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73764f3-8caf-48e5-9d71-778464be0437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ef6b68509e4cc8a40442cf6c158f75)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:01.431755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73764f3-8caf-48e5-9d71-778464be0437 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:01.647635 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:01.662889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e73764f3-8caf-48e5-9d71-778464be0437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ef6b68509e4cc8a40442cf6c158f75', 'request_id': 'req-e73764f3-8caf-48e5-9d71-778464be0437'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:01.687650 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632230e-0db5-4439-9f57-c4b6432b7353 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:01.690152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632230e-0db5-4439-9f57-c4b6432b7353 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:01.691167 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632230e-0db5-4439-9f57-c4b6432b7353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:01.692049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e73764f3-8caf-48e5-9d71-778464be0437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ef6b68509e4cc8a40442cf6c158f75', 'request_id': 'req-e73764f3-8caf-48e5-9d71-778464be0437'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:01.717138 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e73764f3-8caf-48e5-9d71-778464be0437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ef6b68509e4cc8a40442cf6c158f75', 'request_id': 'req-e73764f3-8caf-48e5-9d71-778464be0437'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:01.781770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e73764f3-8caf-48e5-9d71-778464be0437 admin admin] The token cache is being invalidate because project 53ef6b68509e4cc8a40442cf6c158f75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:01.793615 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e73764f3-8caf-48e5-9d71-778464be0437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:01.796616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e73764f3-8caf-48e5-9d71-778464be0437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:01.811948 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 894/1767] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:00 2022] DELETE /identity/v3/projects/53ef6b68509e4cc8a40442cf6c158f75 => generated 0 bytes in 871 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:01.812828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73764f3-8caf-48e5-9d71-778464be0437 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:01.819293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e73764f3-8caf-48e5-9d71-778464be0437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:02.048036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73764f3-8caf-48e5-9d71-778464be0437 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:02.063222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cc236-a05d-4fd5-9abb-3abb66a0f05f tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c90932bb87394c93b81fb7299c26afde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f16a955c714c0681526ec837b834e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:02.064355 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cc236-a05d-4fd5-9abb-3abb66a0f05f tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:02.064903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cc236-a05d-4fd5-9abb-3abb66a0f05f tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:02.065474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cc236-a05d-4fd5-9abb-3abb66a0f05f tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:02.066459 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5cc236-a05d-4fd5-9abb-3abb66a0f05f tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:25:02.067065 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5cc236-a05d-4fd5-9abb-3abb66a0f05f tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:02.069005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5cc236-a05d-4fd5-9abb-3abb66a0f05f tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:02.108680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 895/1768] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:25:02.116427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfb7c6-dc94-4a1a-bbad-9feb43535e80 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:02.117631 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfb7c6-dc94-4a1a-bbad-9feb43535e80 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:02.118911 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfb7c6-dc94-4a1a-bbad-9feb43535e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:02.134711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5632230e-0db5-4439-9f57-c4b6432b7353 None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:02.140948 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5632230e-0db5-4439-9f57-c4b6432b7353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:02.224664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f9dfb7c6-dc94-4a1a-bbad-9feb43535e80 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:02.225220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:02.229390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f9dfb7c6-dc94-4a1a-bbad-9feb43535e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:02.473979 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 896/1769] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:02.473979 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfb7c6-dc94-4a1a-bbad-9feb43535e80 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:02.502679 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfb7c6-dc94-4a1a-bbad-9feb43535e80 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:02.517712 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 874/1770] 10.222.0.88 () {58 vars in 1009 bytes} [Tue Aug 23 22:25:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 836 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:02.523519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbd51a-d35d-498e-adc2-f28e362b6696 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:02.527384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbd51a-d35d-498e-adc2-f28e362b6696 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:02.527384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbd51a-d35d-498e-adc2-f28e362b6696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:02.584052 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49f2d1-6a18-41ca-8bff-bbb98eefa63f tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c90932bb87394c93b81fb7299c26afde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f16a955c714c0681526ec837b834e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:02.586221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49f2d1-6a18-41ca-8bff-bbb98eefa63f tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:02.600579 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49f2d1-6a18-41ca-8bff-bbb98eefa63f tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:02.601912 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49f2d1-6a18-41ca-8bff-bbb98eefa63f tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:02.610556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-a5dbd51a-d35d-498e-adc2-f28e362b6696 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:02.613107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49f2d1-6a18-41ca-8bff-bbb98eefa63f tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:02.613759 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a5dbd51a-d35d-498e-adc2-f28e362b6696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:02.623170 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49f2d1-6a18-41ca-8bff-bbb98eefa63f tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:02.654666 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 875/1771] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:02.658025 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbd51a-d35d-498e-adc2-f28e362b6696 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:02.723960 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbd51a-d35d-498e-adc2-f28e362b6696 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:02.760775 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc319780-a17b-4760-803e-0b06bc00d26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:02.773955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc319780-a17b-4760-803e-0b06bc00d26d admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:02.775090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc319780-a17b-4760-803e-0b06bc00d26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:02.775763 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc319780-a17b-4760-803e-0b06bc00d26d admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:02.780431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc319780-a17b-4760-803e-0b06bc00d26d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:25:02.781874 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc319780-a17b-4760-803e-0b06bc00d26d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:02.791829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc319780-a17b-4760-803e-0b06bc00d26d admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:02.851320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 897/1772] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:02 2022] POST /identity/v3/projects => generated 355 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:02.863008 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49f2d1-6a18-41ca-8bff-bbb98eefa63f tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:02.864188 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 876/1773] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:25:02.870000 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc319780-a17b-4760-803e-0b06bc00d26d admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:02.914459 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc319780-a17b-4760-803e-0b06bc00d26d admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:03.099176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:03.173805 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49f2d1-6a18-41ca-8bff-bbb98eefa63f tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:03.187239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2c0a7-e693-4b09-8801-a429eae04e20 tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c90932bb87394c93b81fb7299c26afde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f16a955c714c0681526ec837b834e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:03.188825 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2c0a7-e693-4b09-8801-a429eae04e20 tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:03.189979 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2c0a7-e693-4b09-8801-a429eae04e20 tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:03.190972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2c0a7-e693-4b09-8801-a429eae04e20 tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:03.193190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2c0a7-e693-4b09-8801-a429eae04e20 tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:25:03.194045 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2c0a7-e693-4b09-8801-a429eae04e20 tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:03.198776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2c0a7-e693-4b09-8801-a429eae04e20 tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:03.237751 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 898/1774] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:25:03.243501 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2c0a7-e693-4b09-8801-a429eae04e20 tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:03.250342 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f9e2c0a7-e693-4b09-8801-a429eae04e20 tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:03.359242 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072981db-d4c8-43f7-ad5b-54ab5ef6fa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:03.360952 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072981db-d4c8-43f7-ad5b-54ab5ef6fa1e admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:03.363224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072981db-d4c8-43f7-ad5b-54ab5ef6fa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:03.364035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072981db-d4c8-43f7-ad5b-54ab5ef6fa1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:03.365801 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072981db-d4c8-43f7-ad5b-54ab5ef6fa1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:03.370308 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072981db-d4c8-43f7-ad5b-54ab5ef6fa1e admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:03.508344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 877/1775] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:02 2022] POST /identity/v3/projects => generated 401 bytes in 644 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:03.511565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072981db-d4c8-43f7-ad5b-54ab5ef6fa1e admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:03.698041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2c0a7-e693-4b09-8801-a429eae04e20 tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:03.822676 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072981db-d4c8-43f7-ad5b-54ab5ef6fa1e admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:03.844236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3385c-9f98-4b28-8966-ddde5ec6d37d tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c90932bb87394c93b81fb7299c26afde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f16a955c714c0681526ec837b834e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:03.845266 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3385c-9f98-4b28-8966-ddde5ec6d37d tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:03.845760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3385c-9f98-4b28-8966-ddde5ec6d37d tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:03.848420 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3385c-9f98-4b28-8966-ddde5ec6d37d tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] PATH_INFO: `/v3/projects/0ce755f3801d453187cb864922f3895a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:03.851795 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3385c-9f98-4b28-8966-ddde5ec6d37d tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0ce755f3801d453187cb864922f3895a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:03.860946 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3385c-9f98-4b28-8966-ddde5ec6d37d tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:03.954345 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:04.094795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb825045-7e3f-44a8-b00c-c2cde3ce5945 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:04.095135 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb825045-7e3f-44a8-b00c-c2cde3ce5945 service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:04.095372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb825045-7e3f-44a8-b00c-c2cde3ce5945 service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:04.095753 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb825045-7e3f-44a8-b00c-c2cde3ce5945 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:04.105490 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-54e3385c-9f98-4b28-8966-ddde5ec6d37d tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce755f3801d453187cb864922f3895a', 'request_id': 'req-54e3385c-9f98-4b28-8966-ddde5ec6d37d'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:04.144388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-54e3385c-9f98-4b28-8966-ddde5ec6d37d tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce755f3801d453187cb864922f3895a', 'request_id': 'req-54e3385c-9f98-4b28-8966-ddde5ec6d37d'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:04.164097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-54e3385c-9f98-4b28-8966-ddde5ec6d37d tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce755f3801d453187cb864922f3895a', 'request_id': 'req-54e3385c-9f98-4b28-8966-ddde5ec6d37d'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:04.194221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-54e3385c-9f98-4b28-8966-ddde5ec6d37d tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] The token cache is being invalidate because project 0ce755f3801d453187cb864922f3895a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:04.194621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-54e3385c-9f98-4b28-8966-ddde5ec6d37d tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:04.197486 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-54e3385c-9f98-4b28-8966-ddde5ec6d37d tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:04.204067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 878/1776] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:03 2022] DELETE /identity/v3/projects/0ce755f3801d453187cb864922f3895a => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:04.209780 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3385c-9f98-4b28-8966-ddde5ec6d37d tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:04.216850 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-54e3385c-9f98-4b28-8966-ddde5ec6d37d tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:04.364646 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb825045-7e3f-44a8-b00c-c2cde3ce5945 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:04.371576 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb825045-7e3f-44a8-b00c-c2cde3ce5945 service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:04.435683 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 899/1777] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:25:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:25:04.573090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3385c-9f98-4b28-8966-ddde5ec6d37d tempest-VolumeQuotaClassesTest-1343852593 tempest-VolumeQuotaClassesTest-1343852593-project-admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:04.590426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74fd75-88ad-4dcc-ab5f-bece6f980648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:04.591430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74fd75-88ad-4dcc-ab5f-bece6f980648 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:04.592040 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74fd75-88ad-4dcc-ab5f-bece6f980648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:04.592284 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74fd75-88ad-4dcc-ab5f-bece6f980648 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:04.594272 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d74fd75-88ad-4dcc-ab5f-bece6f980648 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:04.601748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d74fd75-88ad-4dcc-ab5f-bece6f980648 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:04.720534 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb825045-7e3f-44a8-b00c-c2cde3ce5945 service swift] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:04.722542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 879/1778] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:04 2022] POST /identity/v3/users => generated 301 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:04.738338 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:05.371104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb825045-7e3f-44a8-b00c-c2cde3ce5945 service swift] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:05.399867 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9aebf-4888-4a94-8c3f-c50892151131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:05.400862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9aebf-4888-4a94-8c3f-c50892151131 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:05.403719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9aebf-4888-4a94-8c3f-c50892151131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:05.404255 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9aebf-4888-4a94-8c3f-c50892151131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:05.410135 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9aebf-4888-4a94-8c3f-c50892151131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:05.422587 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9aebf-4888-4a94-8c3f-c50892151131 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:05.488690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 900/1779] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 769 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:25:05.494375 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74fd75-88ad-4dcc-ab5f-bece6f980648 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:05.555222 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74fd75-88ad-4dcc-ab5f-bece6f980648 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:05.614980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565af51f-9439-4700-9a04-38d257b20f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:05.626748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565af51f-9439-4700-9a04-38d257b20f46 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:05.627283 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565af51f-9439-4700-9a04-38d257b20f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:05.639834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565af51f-9439-4700-9a04-38d257b20f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:05.650396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565af51f-9439-4700-9a04-38d257b20f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:05.676194 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565af51f-9439-4700-9a04-38d257b20f46 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:05.784137 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9aebf-4888-4a94-8c3f-c50892151131 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:05.786876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9db9aebf-4888-4a94-8c3f-c50892151131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:05.811716 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 880/1780] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:05.821705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565af51f-9439-4700-9a04-38d257b20f46 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:05.869381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565af51f-9439-4700-9a04-38d257b20f46 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:05.874221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9aebf-4888-4a94-8c3f-c50892151131 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:05.891869 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbf6d5-3a21-4d35-af74-e347dacd444a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:05.893645 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbf6d5-3a21-4d35-af74-e347dacd444a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:05.894331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbf6d5-3a21-4d35-af74-e347dacd444a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:05.894572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973808cc-7cf9-4612-b97d-3ed1fa3d7214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:05.894857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbf6d5-3a21-4d35-af74-e347dacd444a admin admin] PATH_INFO: `/v3/users/ccd386dcbd264ab19a1dd4cbb83d0f2e` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:05.899076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973808cc-7cf9-4612-b97d-3ed1fa3d7214 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:05.899377 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbf6d5-3a21-4d35-af74-e347dacd444a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccd386dcbd264ab19a1dd4cbb83d0f2e)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:05.901364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973808cc-7cf9-4612-b97d-3ed1fa3d7214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:05.901821 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973808cc-7cf9-4612-b97d-3ed1fa3d7214 admin admin] PATH_INFO: `/v3/projects/2dd0b075f5ec431bb8de840c10d3fe50/users/9ed3addd1dda49e5a92a33f530b89e3c/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:05.905853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbf6d5-3a21-4d35-af74-e347dacd444a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:05.991622 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973808cc-7cf9-4612-b97d-3ed1fa3d7214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd0b075f5ec431bb8de840c10d3fe50, user_id=9ed3addd1dda49e5a92a33f530b89e3c, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:06.000147 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973808cc-7cf9-4612-b97d-3ed1fa3d7214 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:06.038112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 881/1781] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:05 2022] PUT /identity/v3/projects/2dd0b075f5ec431bb8de840c10d3fe50/users/9ed3addd1dda49e5a92a33f530b89e3c/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:06.054465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc76d62-0ae3-4f8c-8a49-945ab9529d0c None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:06.055277 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc76d62-0ae3-4f8c-8a49-945ab9529d0c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:06.055788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc76d62-0ae3-4f8c-8a49-945ab9529d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:06.110533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-87cbf6d5-3a21-4d35-af74-e347dacd444a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccd386dcbd264ab19a1dd4cbb83d0f2e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:06.129793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-3dc76d62-0ae3-4f8c-8a49-945ab9529d0c None None] MFA Rules not processed for user `9ed3addd1dda49e5a92a33f530b89e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:06.132097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3dc76d62-0ae3-4f8c-8a49-945ab9529d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:06.185879 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-87cbf6d5-3a21-4d35-af74-e347dacd444a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccd386dcbd264ab19a1dd4cbb83d0f2e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:06.225464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-87cbf6d5-3a21-4d35-af74-e347dacd444a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccd386dcbd264ab19a1dd4cbb83d0f2e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:06.233599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-87cbf6d5-3a21-4d35-af74-e347dacd444a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccd386dcbd264ab19a1dd4cbb83d0f2e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:06.270205 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-87cbf6d5-3a21-4d35-af74-e347dacd444a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccd386dcbd264ab19a1dd4cbb83d0f2e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:06.298477 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 901/1782] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:05 2022] DELETE /identity/v3/users/ccd386dcbd264ab19a1dd4cbb83d0f2e => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:25:06.306293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448937d6-b419-48a0-954b-2e1f0f06a773 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:06.308038 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448937d6-b419-48a0-954b-2e1f0f06a773 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:06.309809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448937d6-b419-48a0-954b-2e1f0f06a773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:06.354391 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:06.395790 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-448937d6-b419-48a0-954b-2e1f0f06a773 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:06.407106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-448937d6-b419-48a0-954b-2e1f0f06a773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:06.426043 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 882/1783] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:06 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:06.430764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc76d62-0ae3-4f8c-8a49-945ab9529d0c None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:06.434574 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3dc76d62-0ae3-4f8c-8a49-945ab9529d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:06.722554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc76d62-0ae3-4f8c-8a49-945ab9529d0c None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:06.741975 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde3703-5933-45e5-8542-54fe02a95572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:06.742701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde3703-5933-45e5-8542-54fe02a95572 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:06.742992 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde3703-5933-45e5-8542-54fe02a95572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:06.743276 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde3703-5933-45e5-8542-54fe02a95572 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:06.743991 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde3703-5933-45e5-8542-54fe02a95572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:06.746346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde3703-5933-45e5-8542-54fe02a95572 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:06.749790 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 902/1784] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:06.751882 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448937d6-b419-48a0-954b-2e1f0f06a773 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:06.766371 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-448937d6-b419-48a0-954b-2e1f0f06a773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:06.812783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448937d6-b419-48a0-954b-2e1f0f06a773 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:06.818212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 883/1785] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:06 2022] POST /identity/v3/projects => generated 401 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:06.833256 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde3703-5933-45e5-8542-54fe02a95572 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:06.836897 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90a824-7711-439b-88e6-d78f692654f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:06.839447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90a824-7711-439b-88e6-d78f692654f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:06.840382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90a824-7711-439b-88e6-d78f692654f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:06.841702 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90a824-7711-439b-88e6-d78f692654f5 admin admin] PATH_INFO: `/v3/users/573b3fe429ac4f5c99918fa610770f57` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:06.847053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90a824-7711-439b-88e6-d78f692654f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=573b3fe429ac4f5c99918fa610770f57)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:06.858152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90a824-7711-439b-88e6-d78f692654f5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:06.880670 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:07.010200 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde3703-5933-45e5-8542-54fe02a95572 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:07.048836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f691b3-498d-4594-aa83-50f231a19089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:07.050311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f691b3-498d-4594-aa83-50f231a19089 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:07.051047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f691b3-498d-4594-aa83-50f231a19089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:07.051678 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f691b3-498d-4594-aa83-50f231a19089 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:07.052860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f691b3-498d-4594-aa83-50f231a19089 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:07.061867 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f691b3-498d-4594-aa83-50f231a19089 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:07.104225 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-cc90a824-7711-439b-88e6-d78f692654f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '573b3fe429ac4f5c99918fa610770f57'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:07.147284 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 884/1786] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:06 2022] POST /identity/v3/users => generated 307 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:07.148614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f691b3-498d-4594-aa83-50f231a19089 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:07.151021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-91f691b3-498d-4594-aa83-50f231a19089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:07.192731 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-cc90a824-7711-439b-88e6-d78f692654f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '573b3fe429ac4f5c99918fa610770f57'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:07.220344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-cc90a824-7711-439b-88e6-d78f692654f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '573b3fe429ac4f5c99918fa610770f57'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:07.222383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-cc90a824-7711-439b-88e6-d78f692654f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '573b3fe429ac4f5c99918fa610770f57'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:07.268124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-cc90a824-7711-439b-88e6-d78f692654f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '573b3fe429ac4f5c99918fa610770f57'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:07.298592 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 903/1787] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:06 2022] DELETE /identity/v3/users/573b3fe429ac4f5c99918fa610770f57 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:07.315774 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b2846-2f42-42b3-9e94-089e41ea239c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:07.316755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b2846-2f42-42b3-9e94-089e41ea239c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:07.317732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b2846-2f42-42b3-9e94-089e41ea239c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:07.333703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f691b3-498d-4594-aa83-50f231a19089 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:07.364300 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fc4b2846-2f42-42b3-9e94-089e41ea239c None None] MFA Rules not processed for user `6505100494d4496591407c48b48535ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:07.366951 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fc4b2846-2f42-42b3-9e94-089e41ea239c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:07.371143 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:07.458615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b681b3-7887-4012-9c7c-614ce6855b7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:07.459449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b681b3-7887-4012-9c7c-614ce6855b7e service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:07.459928 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b681b3-7887-4012-9c7c-614ce6855b7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:07.460589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b681b3-7887-4012-9c7c-614ce6855b7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:07.466622 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-19b681b3-7887-4012-9c7c-614ce6855b7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:07.514362 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 904/1788] 10.222.0.86 () {58 vars in 1015 bytes} [Tue Aug 23 22:25:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:07.517734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b2846-2f42-42b3-9e94-089e41ea239c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:07.525544 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fc4b2846-2f42-42b3-9e94-089e41ea239c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:07.725839 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b2846-2f42-42b3-9e94-089e41ea239c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:07.734879 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21530ccb-9bb4-4cc2-b68d-5e8d68a271b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:07.735955 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21530ccb-9bb4-4cc2-b68d-5e8d68a271b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:07.736734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21530ccb-9bb4-4cc2-b68d-5e8d68a271b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:07.736892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b681b3-7887-4012-9c7c-614ce6855b7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:07.738618 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21530ccb-9bb4-4cc2-b68d-5e8d68a271b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:07.740199 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21530ccb-9bb4-4cc2-b68d-5e8d68a271b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:07.743736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b681b3-7887-4012-9c7c-614ce6855b7e service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:07.744201 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21530ccb-9bb4-4cc2-b68d-5e8d68a271b5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:07.778265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 905/1789] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:07.781912 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a60f14-c327-48b4-b59e-e15b9d388ed9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:07.782833 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a60f14-c327-48b4-b59e-e15b9d388ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:07.783780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a60f14-c327-48b4-b59e-e15b9d388ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:07.787001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 885/1790] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:07.787885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b681b3-7887-4012-9c7c-614ce6855b7e service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:07.803698 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-19b681b3-7887-4012-9c7c-614ce6855b7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:07.830608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-02a60f14-c327-48b4-b59e-e15b9d388ed9 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:07.833403 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-02a60f14-c327-48b4-b59e-e15b9d388ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:07.834017 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b681b3-7887-4012-9c7c-614ce6855b7e service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:07.851544 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 906/1791] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:07.854025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a60f14-c327-48b4-b59e-e15b9d388ed9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:07.863667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c68b7b-eeb6-406b-8dc6-6a303b378e0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:07.864328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c68b7b-eeb6-406b-8dc6-6a303b378e0d service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:07.864643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c68b7b-eeb6-406b-8dc6-6a303b378e0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:07.864977 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c68b7b-eeb6-406b-8dc6-6a303b378e0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:07.880142 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a60f14-c327-48b4-b59e-e15b9d388ed9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:07.891509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ec9b9-4ad3-408d-b29b-75c1213382e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:07.893736 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ec9b9-4ad3-408d-b29b-75c1213382e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:07.894847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ec9b9-4ad3-408d-b29b-75c1213382e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:07.895852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ec9b9-4ad3-408d-b29b-75c1213382e3 admin admin] PATH_INFO: `/v3/projects/755bbed556204670891cdbb630076bd0/users/74982ae23b8a454f863b74de64557638/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:07.905878 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c68b7b-eeb6-406b-8dc6-6a303b378e0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:07.913098 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c68b7b-eeb6-406b-8dc6-6a303b378e0d service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:07.932028 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ec9b9-4ad3-408d-b29b-75c1213382e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=755bbed556204670891cdbb630076bd0, user_id=74982ae23b8a454f863b74de64557638, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:07.939577 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ec9b9-4ad3-408d-b29b-75c1213382e3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:07.957133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:07.961287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 886/1792] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:07.975263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 907/1793] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:07 2022] PUT /identity/v3/projects/755bbed556204670891cdbb630076bd0/users/74982ae23b8a454f863b74de64557638/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:07.983641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c68b7b-eeb6-406b-8dc6-6a303b378e0d service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:08.012300 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c68b7b-eeb6-406b-8dc6-6a303b378e0d service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:08.153958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f030ade-b2fd-43bf-8b49-8d876065ddcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:08.155778 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f030ade-b2fd-43bf-8b49-8d876065ddcb admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:08.156572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f030ade-b2fd-43bf-8b49-8d876065ddcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:08.157563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f030ade-b2fd-43bf-8b49-8d876065ddcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:08.160202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f030ade-b2fd-43bf-8b49-8d876065ddcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:08.164607 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f030ade-b2fd-43bf-8b49-8d876065ddcb admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:08.195672 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ec9b9-4ad3-408d-b29b-75c1213382e3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:08.222948 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 887/1794] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:08.226647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f030ade-b2fd-43bf-8b49-8d876065ddcb admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:08.278117 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f030ade-b2fd-43bf-8b49-8d876065ddcb admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:08.310899 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5b88f-3ec5-4bff-b341-a2a10c93045e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:08.312787 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5b88f-3ec5-4bff-b341-a2a10c93045e admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:08.314387 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5b88f-3ec5-4bff-b341-a2a10c93045e admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:08.314891 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5b88f-3ec5-4bff-b341-a2a10c93045e admin admin] PATH_INFO: `/v3/projects/755bbed556204670891cdbb630076bd0/users/74982ae23b8a454f863b74de64557638/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:08.322319 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5b88f-3ec5-4bff-b341-a2a10c93045e admin admin] RBAC: Authorizing `identity:create_grant(project_id=755bbed556204670891cdbb630076bd0, user_id=74982ae23b8a454f863b74de64557638, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:08.331127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5b88f-3ec5-4bff-b341-a2a10c93045e admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:08.388619 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 888/1795] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:08 2022] PUT /identity/v3/projects/755bbed556204670891cdbb630076bd0/users/74982ae23b8a454f863b74de64557638/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:08.392706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab7ad1-2f80-4086-993a-979c10043652 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:08.394366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab7ad1-2f80-4086-993a-979c10043652 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:08.395211 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab7ad1-2f80-4086-993a-979c10043652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:08.478733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ec9b9-4ad3-408d-b29b-75c1213382e3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:08.554002 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-6dab7ad1-2f80-4086-993a-979c10043652 None None] MFA Rules not processed for user `74982ae23b8a454f863b74de64557638`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:08.555573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6dab7ad1-2f80-4086-993a-979c10043652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:08.682074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb1a07-9dfb-49c9-a7e4-b81684f92689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:08.682970 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:08.683195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb1a07-9dfb-49c9-a7e4-b81684f92689 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:08.686429 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb1a07-9dfb-49c9-a7e4-b81684f92689 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:08.686881 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb1a07-9dfb-49c9-a7e4-b81684f92689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:08.939838 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 889/1796] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:08 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 551 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:08.971529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab7ad1-2f80-4086-993a-979c10043652 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:08.974584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6dab7ad1-2f80-4086-993a-979c10043652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:09.086852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb1a07-9dfb-49c9-a7e4-b81684f92689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:09.094196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb1a07-9dfb-49c9-a7e4-b81684f92689 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:09.193884 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 908/1797] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 999 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:09.239410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab7ad1-2f80-4086-993a-979c10043652 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:09.251778 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13a36f-2ff3-4444-bb88-ebc08469b78b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:09.253771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13a36f-2ff3-4444-bb88-ebc08469b78b service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:09.254371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13a36f-2ff3-4444-bb88-ebc08469b78b service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:09.254892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13a36f-2ff3-4444-bb88-ebc08469b78b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:09.274418 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13a36f-2ff3-4444-bb88-ebc08469b78b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:09.277025 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13a36f-2ff3-4444-bb88-ebc08469b78b service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:09.322091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 890/1798] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:08 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:09.582843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb1a07-9dfb-49c9-a7e4-b81684f92689 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:09.620491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb1a07-9dfb-49c9-a7e4-b81684f92689 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:09.646077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7c012-84f2-4514-8025-7b3a8ae62064 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:09.647717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7c012-84f2-4514-8025-7b3a8ae62064 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:09.648550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7c012-84f2-4514-8025-7b3a8ae62064 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:09.649802 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7c012-84f2-4514-8025-7b3a8ae62064 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:09.656517 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d0b7c012-84f2-4514-8025-7b3a8ae62064 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:09.893931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7c012-84f2-4514-8025-7b3a8ae62064 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:09.903888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7c012-84f2-4514-8025-7b3a8ae62064 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:09.944554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 909/1799] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:09 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:25:10.533889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13a36f-2ff3-4444-bb88-ebc08469b78b service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:10.538380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6b13a36f-2ff3-4444-bb88-ebc08469b78b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:10.580945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13a36f-2ff3-4444-bb88-ebc08469b78b service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:10.596593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc44e3-cd50-451d-a881-6a0bce11c59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:10.598978 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc44e3-cd50-451d-a881-6a0bce11c59a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:10.599362 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc44e3-cd50-451d-a881-6a0bce11c59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:10.599736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc44e3-cd50-451d-a881-6a0bce11c59a admin admin] PATH_INFO: `/v3/projects/93a4338bace34535aac26747f4e8d69c` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:10.602893 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc44e3-cd50-451d-a881-6a0bce11c59a admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a4338bace34535aac26747f4e8d69c)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:10.610071 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc44e3-cd50-451d-a881-6a0bce11c59a admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:10.882392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-20cc44e3-cd50-451d-a881-6a0bce11c59a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a4338bace34535aac26747f4e8d69c', 'request_id': 'req-20cc44e3-cd50-451d-a881-6a0bce11c59a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:10.908043 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-20cc44e3-cd50-451d-a881-6a0bce11c59a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a4338bace34535aac26747f4e8d69c', 'request_id': 'req-20cc44e3-cd50-451d-a881-6a0bce11c59a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:10.925218 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-20cc44e3-cd50-451d-a881-6a0bce11c59a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a4338bace34535aac26747f4e8d69c', 'request_id': 'req-20cc44e3-cd50-451d-a881-6a0bce11c59a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:10.965027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-20cc44e3-cd50-451d-a881-6a0bce11c59a admin admin] The token cache is being invalidate because project 93a4338bace34535aac26747f4e8d69c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:10.965458 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-20cc44e3-cd50-451d-a881-6a0bce11c59a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:10.972582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-20cc44e3-cd50-451d-a881-6a0bce11c59a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:10.982126 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 891/1800] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:10 2022] DELETE /identity/v3/projects/93a4338bace34535aac26747f4e8d69c => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:10.984446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7c012-84f2-4514-8025-7b3a8ae62064 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:10.994790 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:11.030780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7c012-84f2-4514-8025-7b3a8ae62064 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:11.391043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b17c73-e393-4ca1-9a45-1ee7c5121b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:11.391987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b17c73-e393-4ca1-9a45-1ee7c5121b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:11.392502 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b17c73-e393-4ca1-9a45-1ee7c5121b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:11.392927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b17c73-e393-4ca1-9a45-1ee7c5121b1e admin admin] PATH_INFO: `/v3/users/c90932bb87394c93b81fb7299c26afde` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:11.394753 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b17c73-e393-4ca1-9a45-1ee7c5121b1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c90932bb87394c93b81fb7299c26afde)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:11.397545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b17c73-e393-4ca1-9a45-1ee7c5121b1e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:11.569675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e5b17c73-e393-4ca1-9a45-1ee7c5121b1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c90932bb87394c93b81fb7299c26afde'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:11.614008 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e5b17c73-e393-4ca1-9a45-1ee7c5121b1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c90932bb87394c93b81fb7299c26afde'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:11.635937 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e5b17c73-e393-4ca1-9a45-1ee7c5121b1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c90932bb87394c93b81fb7299c26afde'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:11.641725 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e5b17c73-e393-4ca1-9a45-1ee7c5121b1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c90932bb87394c93b81fb7299c26afde'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:11.708791 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e5b17c73-e393-4ca1-9a45-1ee7c5121b1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c90932bb87394c93b81fb7299c26afde'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:11.754423 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 910/1801] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:10 2022] DELETE /identity/v3/users/c90932bb87394c93b81fb7299c26afde => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:12.915039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc44e3-cd50-451d-a881-6a0bce11c59a admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:12.918169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-20cc44e3-cd50-451d-a881-6a0bce11c59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:12.951281 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc44e3-cd50-451d-a881-6a0bce11c59a admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:13.097265 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:13.119525 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f71f89-2488-499b-8293-1445f78540c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:13.121751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f71f89-2488-499b-8293-1445f78540c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:13.123430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f71f89-2488-499b-8293-1445f78540c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:13.123921 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f71f89-2488-499b-8293-1445f78540c9 admin admin] PATH_INFO: `/v3/projects/df2149b69f854ab2a2e094e51148b3b6` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:13.126490 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f71f89-2488-499b-8293-1445f78540c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df2149b69f854ab2a2e094e51148b3b6)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:13.132669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f71f89-2488-499b-8293-1445f78540c9 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:13.227351 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e2f71f89-2488-499b-8293-1445f78540c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df2149b69f854ab2a2e094e51148b3b6', 'request_id': 'req-e2f71f89-2488-499b-8293-1445f78540c9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:13.237344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e2f71f89-2488-499b-8293-1445f78540c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df2149b69f854ab2a2e094e51148b3b6', 'request_id': 'req-e2f71f89-2488-499b-8293-1445f78540c9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:13.246139 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e2f71f89-2488-499b-8293-1445f78540c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df2149b69f854ab2a2e094e51148b3b6', 'request_id': 'req-e2f71f89-2488-499b-8293-1445f78540c9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:13.275178 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e2f71f89-2488-499b-8293-1445f78540c9 admin admin] The token cache is being invalidate because project df2149b69f854ab2a2e094e51148b3b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:13.276055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e2f71f89-2488-499b-8293-1445f78540c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:13.277909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e2f71f89-2488-499b-8293-1445f78540c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:13.282406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 892/1802] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:12 2022] DELETE /identity/v3/projects/df2149b69f854ab2a2e094e51148b3b6 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:25:13.287790 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b17c73-e393-4ca1-9a45-1ee7c5121b1e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:13.290187 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e5b17c73-e393-4ca1-9a45-1ee7c5121b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:13.333426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b17c73-e393-4ca1-9a45-1ee7c5121b1e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:13.559479 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fdba1-7a83-49cc-8710-85ccd81c2eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:13.561902 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fdba1-7a83-49cc-8710-85ccd81c2eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:13.562486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fdba1-7a83-49cc-8710-85ccd81c2eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:13.563037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fdba1-7a83-49cc-8710-85ccd81c2eaf admin admin] PATH_INFO: `/v3/users/b4ca49c8a2cf4575a326df65ef5d0bd4` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:13.564697 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fdba1-7a83-49cc-8710-85ccd81c2eaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4ca49c8a2cf4575a326df65ef5d0bd4)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:13.568559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fdba1-7a83-49cc-8710-85ccd81c2eaf admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:13.616238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:13.678569 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-954fdba1-7a83-49cc-8710-85ccd81c2eaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4ca49c8a2cf4575a326df65ef5d0bd4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:13.699537 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-954fdba1-7a83-49cc-8710-85ccd81c2eaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4ca49c8a2cf4575a326df65ef5d0bd4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:13.716565 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-954fdba1-7a83-49cc-8710-85ccd81c2eaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4ca49c8a2cf4575a326df65ef5d0bd4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:13.719189 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-954fdba1-7a83-49cc-8710-85ccd81c2eaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4ca49c8a2cf4575a326df65ef5d0bd4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:13.768709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-954fdba1-7a83-49cc-8710-85ccd81c2eaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4ca49c8a2cf4575a326df65ef5d0bd4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:13.811367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 911/1803] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:13 2022] DELETE /identity/v3/users/b4ca49c8a2cf4575a326df65ef5d0bd4 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:14.400967 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3c790-1f12-4132-b12b-1abd1241b19f None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:14.405663 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3c790-1f12-4132-b12b-1abd1241b19f None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:14.406216 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3c790-1f12-4132-b12b-1abd1241b19f None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:14.408062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 893/1804] 10.222.0.88 () {54 vars in 921 bytes} [Tue Aug 23 22:25:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:25:14.561783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e54afb-7aac-47f3-a1e6-6de0f20df897 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:14.562450 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e54afb-7aac-47f3-a1e6-6de0f20df897 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:14.562833 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e54afb-7aac-47f3-a1e6-6de0f20df897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:14.719673 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-80e54afb-7aac-47f3-a1e6-6de0f20df897 None None] MFA Rules not processed for user `e3517b3b077a4a1fa76163657b46930c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:14.721644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-80e54afb-7aac-47f3-a1e6-6de0f20df897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:14.875539 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 912/1805] 10.222.0.88 () {58 vars in 1018 bytes} [Tue Aug 23 22:25:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:15.729454 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3c790-1f12-4132-b12b-1abd1241b19f None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:15.733464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f2c3c790-1f12-4132-b12b-1abd1241b19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:15.964598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3c790-1f12-4132-b12b-1abd1241b19f None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:15.977285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0cd8e-e5d0-4a8f-9dae-7175a335d55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:15.978660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0cd8e-e5d0-4a8f-9dae-7175a335d55b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:15.979123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0cd8e-e5d0-4a8f-9dae-7175a335d55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:15.980931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0cd8e-e5d0-4a8f-9dae-7175a335d55b admin admin] PATH_INFO: `/v3/projects/a5f16a955c714c0681526ec837b834e7` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:15.985310 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0cd8e-e5d0-4a8f-9dae-7175a335d55b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5f16a955c714c0681526ec837b834e7)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:15.998977 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0cd8e-e5d0-4a8f-9dae-7175a335d55b admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:16.069021 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e54afb-7aac-47f3-a1e6-6de0f20df897 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:16.071890 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-80e54afb-7aac-47f3-a1e6-6de0f20df897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:16.124666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e54afb-7aac-47f3-a1e6-6de0f20df897 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:16.150639 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca619b9-e799-4316-8990-db6fd28a4b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:16.152229 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca619b9-e799-4316-8990-db6fd28a4b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:16.156552 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca619b9-e799-4316-8990-db6fd28a4b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:16.157031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca619b9-e799-4316-8990-db6fd28a4b31 admin admin] PATH_INFO: `/v3/users/9ed3addd1dda49e5a92a33f530b89e3c` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:16.165259 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca619b9-e799-4316-8990-db6fd28a4b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ed3addd1dda49e5a92a33f530b89e3c)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:16.175815 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca619b9-e799-4316-8990-db6fd28a4b31 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:16.237109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4ad0cd8e-e5d0-4a8f-9dae-7175a335d55b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5f16a955c714c0681526ec837b834e7', 'request_id': 'req-4ad0cd8e-e5d0-4a8f-9dae-7175a335d55b'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:16.283346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4ad0cd8e-e5d0-4a8f-9dae-7175a335d55b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5f16a955c714c0681526ec837b834e7', 'request_id': 'req-4ad0cd8e-e5d0-4a8f-9dae-7175a335d55b'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:16.321536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4ad0cd8e-e5d0-4a8f-9dae-7175a335d55b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5f16a955c714c0681526ec837b834e7', 'request_id': 'req-4ad0cd8e-e5d0-4a8f-9dae-7175a335d55b'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:16.357107 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4ad0cd8e-e5d0-4a8f-9dae-7175a335d55b admin admin] The token cache is being invalidate because project a5f16a955c714c0681526ec837b834e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:16.363744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4ad0cd8e-e5d0-4a8f-9dae-7175a335d55b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:16.365429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4ad0cd8e-e5d0-4a8f-9dae-7175a335d55b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:16.376760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 894/1806] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:15 2022] DELETE /identity/v3/projects/a5f16a955c714c0681526ec837b834e7 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:16.379724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4beea28-6d14-4ed7-ada7-4fbb3ee2e8fe None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:16.380324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4beea28-6d14-4ed7-ada7-4fbb3ee2e8fe None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:16.380917 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4beea28-6d14-4ed7-ada7-4fbb3ee2e8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:16.390256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ca619b9-e799-4316-8990-db6fd28a4b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ed3addd1dda49e5a92a33f530b89e3c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:16.442069 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-a4beea28-6d14-4ed7-ada7-4fbb3ee2e8fe None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:16.444570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a4beea28-6d14-4ed7-ada7-4fbb3ee2e8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:16.456677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:16.478798 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ca619b9-e799-4316-8990-db6fd28a4b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ed3addd1dda49e5a92a33f530b89e3c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:16.511566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ca619b9-e799-4316-8990-db6fd28a4b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ed3addd1dda49e5a92a33f530b89e3c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:16.513595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ca619b9-e799-4316-8990-db6fd28a4b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ed3addd1dda49e5a92a33f530b89e3c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:16.541699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ca619b9-e799-4316-8990-db6fd28a4b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ed3addd1dda49e5a92a33f530b89e3c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:16.575908 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 913/1807] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:16 2022] DELETE /identity/v3/users/9ed3addd1dda49e5a92a33f530b89e3c => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:16.581676 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96447d2e-5612-480e-920f-88d674eda076 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:16.586469 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96447d2e-5612-480e-920f-88d674eda076 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:16.587382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96447d2e-5612-480e-920f-88d674eda076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:16.624309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:16.645607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-96447d2e-5612-480e-920f-88d674eda076 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:16.650125 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-96447d2e-5612-480e-920f-88d674eda076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:16.724957 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 895/1808] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:16.727021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4beea28-6d14-4ed7-ada7-4fbb3ee2e8fe None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:16.735058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a4beea28-6d14-4ed7-ada7-4fbb3ee2e8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:16.916920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 914/1809] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:16.919529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb8195-647b-4175-9beb-17a49d413f3f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:16.920017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb8195-647b-4175-9beb-17a49d413f3f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:16.920295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb8195-647b-4175-9beb-17a49d413f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:16.973920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fffb8195-647b-4175-9beb-17a49d413f3f None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:16.975233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fffb8195-647b-4175-9beb-17a49d413f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:17.018814 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 915/1810] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:17.020646 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb8195-647b-4175-9beb-17a49d413f3f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:17.022122 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fffb8195-647b-4175-9beb-17a49d413f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:17.039066 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4beea28-6d14-4ed7-ada7-4fbb3ee2e8fe None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:17.223790 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb8195-647b-4175-9beb-17a49d413f3f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:17.242297 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d5b03-313d-438f-a852-9706776bb497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:17.245860 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d5b03-313d-438f-a852-9706776bb497 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:17.248818 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d5b03-313d-438f-a852-9706776bb497 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:17.249737 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d5b03-313d-438f-a852-9706776bb497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:17.291735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d5b03-313d-438f-a852-9706776bb497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:17.296106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d5b03-313d-438f-a852-9706776bb497 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:17.327512 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1309a-97fd-46c7-ade1-166506447c74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:17.331642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1309a-97fd-46c7-ade1-166506447c74 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:17.334685 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1309a-97fd-46c7-ade1-166506447c74 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:17.335332 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1309a-97fd-46c7-ade1-166506447c74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:17.342686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-77c1309a-97fd-46c7-ade1-166506447c74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:17.350171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 916/1811] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:25:17.355891 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d5b03-313d-438f-a852-9706776bb497 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:17.396263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d5b03-313d-438f-a852-9706776bb497 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:17.401679 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1309a-97fd-46c7-ade1-166506447c74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:17.408553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1309a-97fd-46c7-ade1-166506447c74 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:17.425229 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244ae1a-2a3b-47b9-a093-5de93762310b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:17.426309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244ae1a-2a3b-47b9-a093-5de93762310b admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:17.426820 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244ae1a-2a3b-47b9-a093-5de93762310b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:17.427328 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244ae1a-2a3b-47b9-a093-5de93762310b admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:17.428778 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244ae1a-2a3b-47b9-a093-5de93762310b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:25:17.429409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244ae1a-2a3b-47b9-a093-5de93762310b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:17.434972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244ae1a-2a3b-47b9-a093-5de93762310b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:17.451476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 896/1812] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:17.489883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1309a-97fd-46c7-ade1-166506447c74 service nova] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:17.505560 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 917/1813] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:25:17.509752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1309a-97fd-46c7-ade1-166506447c74 service nova] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:17.521436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef17ad-acfc-49bf-add6-d3df5c4885ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:17.525044 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef17ad-acfc-49bf-add6-d3df5c4885ad admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:17.525726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef17ad-acfc-49bf-add6-d3df5c4885ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:17.526285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef17ad-acfc-49bf-add6-d3df5c4885ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:17.527382 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef17ad-acfc-49bf-add6-d3df5c4885ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:17.533890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef17ad-acfc-49bf-add6-d3df5c4885ad admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:17.646515 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244ae1a-2a3b-47b9-a093-5de93762310b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:17.647373 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 897/1814] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:17 2022] POST /identity/v3/projects => generated 359 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:17.677996 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244ae1a-2a3b-47b9-a093-5de93762310b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:17.836298 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e5f1e-7313-4b14-9eff-7c3b6fa3f9a8 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:17.836921 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e5f1e-7313-4b14-9eff-7c3b6fa3f9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:17.836921 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e5f1e-7313-4b14-9eff-7c3b6fa3f9a8 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:17.838811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 898/1815] 10.222.0.88 () {54 vars in 921 bytes} [Tue Aug 23 22:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:25:17.882439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646979c3-d1fd-4721-9c56-c750443b92a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:17.883308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646979c3-d1fd-4721-9c56-c750443b92a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:17.883649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646979c3-d1fd-4721-9c56-c750443b92a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:17.883933 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646979c3-d1fd-4721-9c56-c750443b92a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:17.884733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646979c3-d1fd-4721-9c56-c750443b92a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:17.887014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646979c3-d1fd-4721-9c56-c750443b92a6 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:17.953180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 918/1816] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:17 2022] POST /identity/v3/users => generated 280 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:25:17.958613 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e5f1e-7313-4b14-9eff-7c3b6fa3f9a8 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:17.987535 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e5f1e-7313-4b14-9eff-7c3b6fa3f9a8 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:17.998708 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b8e8d-1783-4fae-9121-8a370a787d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:18.000695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b8e8d-1783-4fae-9121-8a370a787d84 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:18.000695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b8e8d-1783-4fae-9121-8a370a787d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:18.001290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b8e8d-1783-4fae-9121-8a370a787d84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:18.002795 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b8e8d-1783-4fae-9121-8a370a787d84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:18.006614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b8e8d-1783-4fae-9121-8a370a787d84 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:18.041320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a7a50-04cc-4341-a933-d6f95fdd8d8f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:18.042049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a7a50-04cc-4341-a933-d6f95fdd8d8f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:18.042725 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a7a50-04cc-4341-a933-d6f95fdd8d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:18.049834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 899/1817] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:18.053854 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b8e8d-1783-4fae-9121-8a370a787d84 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:18.075791 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:18.101319 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b8e8d-1783-4fae-9121-8a370a787d84 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:18.121356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad26470-65be-4c1e-8df3-5e96c6c94fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:18.123678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-352a7a50-04cc-4341-a933-d6f95fdd8d8f None None] MFA Rules not processed for user `e3517b3b077a4a1fa76163657b46930c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:18.126298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-352a7a50-04cc-4341-a933-d6f95fdd8d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:18.128006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad26470-65be-4c1e-8df3-5e96c6c94fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:18.128744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad26470-65be-4c1e-8df3-5e96c6c94fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:18.129455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad26470-65be-4c1e-8df3-5e96c6c94fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:18.131544 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad26470-65be-4c1e-8df3-5e96c6c94fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:18.136153 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad26470-65be-4c1e-8df3-5e96c6c94fa8 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:18.175774 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 900/1818] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:18.177322 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad26470-65be-4c1e-8df3-5e96c6c94fa8 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:18.183428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:18.235877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad26470-65be-4c1e-8df3-5e96c6c94fa8 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:18.264556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fefd2d-f844-4aeb-b234-bcffe672a5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:18.266615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fefd2d-f844-4aeb-b234-bcffe672a5f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:18.266860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fefd2d-f844-4aeb-b234-bcffe672a5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:18.267567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fefd2d-f844-4aeb-b234-bcffe672a5f1 admin admin] PATH_INFO: `/v3/projects/3e9e7d6ad24542ccb6ee81fea23ecf21/users/0fdfd369a8764445a1016800ff595d30/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:18.322013 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fefd2d-f844-4aeb-b234-bcffe672a5f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e9e7d6ad24542ccb6ee81fea23ecf21, user_id=0fdfd369a8764445a1016800ff595d30, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:18.330515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fefd2d-f844-4aeb-b234-bcffe672a5f1 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:18.368306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 901/1819] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:18 2022] PUT /identity/v3/projects/3e9e7d6ad24542ccb6ee81fea23ecf21/users/0fdfd369a8764445a1016800ff595d30/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:18.397255 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a6e38-7386-4383-b56a-c6922b19daf1 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:18.397672 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a6e38-7386-4383-b56a-c6922b19daf1 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:18.398311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a6e38-7386-4383-b56a-c6922b19daf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:18.434842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 919/1820] 10.222.0.88 () {58 vars in 1018 bytes} [Tue Aug 23 22:25:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:18.460706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-479a6e38-7386-4383-b56a-c6922b19daf1 None None] MFA Rules not processed for user `0fdfd369a8764445a1016800ff595d30`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:18.462424 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-479a6e38-7386-4383-b56a-c6922b19daf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:18.589719 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 902/1821] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:18.649269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a7a50-04cc-4341-a933-d6f95fdd8d8f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:18.651180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-352a7a50-04cc-4341-a933-d6f95fdd8d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:18.792367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a7a50-04cc-4341-a933-d6f95fdd8d8f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:18.998985 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33af10-d256-4395-b1d1-c6701867aade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:18.999890 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33af10-d256-4395-b1d1-c6701867aade service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:19.000205 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33af10-d256-4395-b1d1-c6701867aade service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:19.000563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33af10-d256-4395-b1d1-c6701867aade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:19.060568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33af10-d256-4395-b1d1-c6701867aade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:19.064064 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33af10-d256-4395-b1d1-c6701867aade service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:19.106937 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 920/1822] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:18 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:25:19.127708 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a6e38-7386-4383-b56a-c6922b19daf1 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:19.263053 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a6e38-7386-4383-b56a-c6922b19daf1 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:19.272301 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74185e88-f9cc-4d48-9764-1fb60eb89b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:19.273556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74185e88-f9cc-4d48-9764-1fb60eb89b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:19.273906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74185e88-f9cc-4d48-9764-1fb60eb89b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:19.274408 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74185e88-f9cc-4d48-9764-1fb60eb89b20 admin admin] PATH_INFO: `/v3/users/0fdfd369a8764445a1016800ff595d30` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:19.276689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74185e88-f9cc-4d48-9764-1fb60eb89b20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fdfd369a8764445a1016800ff595d30)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:19.280936 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74185e88-f9cc-4d48-9764-1fb60eb89b20 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:19.452137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-74185e88-f9cc-4d48-9764-1fb60eb89b20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fdfd369a8764445a1016800ff595d30'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:19.453173 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-74185e88-f9cc-4d48-9764-1fb60eb89b20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fdfd369a8764445a1016800ff595d30'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:19.473903 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-74185e88-f9cc-4d48-9764-1fb60eb89b20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fdfd369a8764445a1016800ff595d30'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:19.484804 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:19.509306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-74185e88-f9cc-4d48-9764-1fb60eb89b20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fdfd369a8764445a1016800ff595d30'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:19.531258 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-74185e88-f9cc-4d48-9764-1fb60eb89b20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fdfd369a8764445a1016800ff595d30'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:19.542083 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 903/1823] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:19 2022] DELETE /identity/v3/users/0fdfd369a8764445a1016800ff595d30 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:19.550359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac39c5-64c9-4a1e-a1be-c1aba28e9c18 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:19.551457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac39c5-64c9-4a1e-a1be-c1aba28e9c18 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:19.551985 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac39c5-64c9-4a1e-a1be-c1aba28e9c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:19.635175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ddac39c5-64c9-4a1e-a1be-c1aba28e9c18 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:19.638181 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ddac39c5-64c9-4a1e-a1be-c1aba28e9c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:19.791309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 921/1824] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:19.861961 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74185e88-f9cc-4d48-9764-1fb60eb89b20 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:19.864471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-74185e88-f9cc-4d48-9764-1fb60eb89b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:20.082885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74185e88-f9cc-4d48-9764-1fb60eb89b20 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:20.099948 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa5d22-580a-4ed2-a96b-8df1ec3fc6a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:20.103864 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa5d22-580a-4ed2-a96b-8df1ec3fc6a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:20.105113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa5d22-580a-4ed2-a96b-8df1ec3fc6a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:20.107682 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa5d22-580a-4ed2-a96b-8df1ec3fc6a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:20.165615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aa5d22-580a-4ed2-a96b-8df1ec3fc6a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:20.172045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aa5d22-580a-4ed2-a96b-8df1ec3fc6a5 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:20.185623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac39c5-64c9-4a1e-a1be-c1aba28e9c18 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:20.187650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ddac39c5-64c9-4a1e-a1be-c1aba28e9c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:20.197734 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 904/1825] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:20.207893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:20.539367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac39c5-64c9-4a1e-a1be-c1aba28e9c18 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:20.545566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa5d22-580a-4ed2-a96b-8df1ec3fc6a5 service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:20.549557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-84aa5d22-580a-4ed2-a96b-8df1ec3fc6a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:20.571052 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b76133-ac30-4b2c-b88f-4d1060b4723b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:20.571961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b76133-ac30-4b2c-b88f-4d1060b4723b service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:20.572128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b76133-ac30-4b2c-b88f-4d1060b4723b service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:20.572316 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b76133-ac30-4b2c-b88f-4d1060b4723b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:20.575249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-59b76133-ac30-4b2c-b88f-4d1060b4723b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:20.579513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa5d22-580a-4ed2-a96b-8df1ec3fc6a5 service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:20.601694 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:20.603774 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:20.604543 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:20.608036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb admin admin] PATH_INFO: `/v3/projects/2dd0b075f5ec431bb8de840c10d3fe50` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:20.611728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd0b075f5ec431bb8de840c10d3fe50)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:20.616292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b76133-ac30-4b2c-b88f-4d1060b4723b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:20.616885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:20.628603 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b76133-ac30-4b2c-b88f-4d1060b4723b service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:20.665142 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 922/1826] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:20.858471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd0b075f5ec431bb8de840c10d3fe50', 'request_id': 'req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:20.868072 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd0b075f5ec431bb8de840c10d3fe50', 'request_id': 'req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:20.878248 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd0b075f5ec431bb8de840c10d3fe50', 'request_id': 'req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:20.902432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb admin admin] The token cache is being invalidate because project 2dd0b075f5ec431bb8de840c10d3fe50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:20.903875 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:20.905290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:20.914296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b76133-ac30-4b2c-b88f-4d1060b4723b service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:20.923598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 905/1827] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:20 2022] DELETE /identity/v3/projects/2dd0b075f5ec431bb8de840c10d3fe50 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:20.931526 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b76133-ac30-4b2c-b88f-4d1060b4723b service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:21.048296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e38486-1c5c-4efd-8bff-14c1d6470fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:21.049510 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e38486-1c5c-4efd-8bff-14c1d6470fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:21.049510 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e38486-1c5c-4efd-8bff-14c1d6470fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:21.049855 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e38486-1c5c-4efd-8bff-14c1d6470fbd admin admin] PATH_INFO: `/v3/users/74982ae23b8a454f863b74de64557638` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:21.054341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e38486-1c5c-4efd-8bff-14c1d6470fbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=74982ae23b8a454f863b74de64557638)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:21.056720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e38486-1c5c-4efd-8bff-14c1d6470fbd admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:21.165685 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d1e38486-1c5c-4efd-8bff-14c1d6470fbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74982ae23b8a454f863b74de64557638'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:21.191486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d1e38486-1c5c-4efd-8bff-14c1d6470fbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74982ae23b8a454f863b74de64557638'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:21.207692 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d1e38486-1c5c-4efd-8bff-14c1d6470fbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74982ae23b8a454f863b74de64557638'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:21.208595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d1e38486-1c5c-4efd-8bff-14c1d6470fbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74982ae23b8a454f863b74de64557638'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:21.222766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d1e38486-1c5c-4efd-8bff-14c1d6470fbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74982ae23b8a454f863b74de64557638'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:21.235654 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 923/1828] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:20 2022] DELETE /identity/v3/users/74982ae23b8a454f863b74de64557638 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:21.487541 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:21.491992 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:21.516954 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97520b-8e7d-496d-bfb4-61caf5ba8ddb admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:21.660414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a27a9-224e-4fc0-857f-b58fde88c2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:21.661631 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a27a9-224e-4fc0-857f-b58fde88c2ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:21.662195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a27a9-224e-4fc0-857f-b58fde88c2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:21.662761 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a27a9-224e-4fc0-857f-b58fde88c2ec admin admin] PATH_INFO: `/v3/projects/01a3ab7444bb440189417eb5dcedf204` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:21.665262 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a27a9-224e-4fc0-857f-b58fde88c2ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=01a3ab7444bb440189417eb5dcedf204)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:21.670770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a27a9-224e-4fc0-857f-b58fde88c2ec admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:21.676197 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:21.832185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5d6a27a9-224e-4fc0-857f-b58fde88c2ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01a3ab7444bb440189417eb5dcedf204', 'request_id': 'req-5d6a27a9-224e-4fc0-857f-b58fde88c2ec'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:21.845513 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e38486-1c5c-4efd-8bff-14c1d6470fbd admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:21.847454 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d1e38486-1c5c-4efd-8bff-14c1d6470fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:21.853726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5d6a27a9-224e-4fc0-857f-b58fde88c2ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01a3ab7444bb440189417eb5dcedf204', 'request_id': 'req-5d6a27a9-224e-4fc0-857f-b58fde88c2ec'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:21.870684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5d6a27a9-224e-4fc0-857f-b58fde88c2ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01a3ab7444bb440189417eb5dcedf204', 'request_id': 'req-5d6a27a9-224e-4fc0-857f-b58fde88c2ec'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:21.874718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e38486-1c5c-4efd-8bff-14c1d6470fbd admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:21.889230 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:21.907121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5d6a27a9-224e-4fc0-857f-b58fde88c2ec admin admin] The token cache is being invalidate because project 01a3ab7444bb440189417eb5dcedf204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:21.907926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5d6a27a9-224e-4fc0-857f-b58fde88c2ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:21.910642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5d6a27a9-224e-4fc0-857f-b58fde88c2ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:21.915948 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 906/1829] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:21 2022] DELETE /identity/v3/projects/01a3ab7444bb440189417eb5dcedf204 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:21.938078 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0cfe9-1d1a-4c33-b2d5-759a7dfc4e8b None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:21.939390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0cfe9-1d1a-4c33-b2d5-759a7dfc4e8b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:21.940170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0cfe9-1d1a-4c33-b2d5-759a7dfc4e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:21.971341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82123c-82cd-4400-b796-ac143e14ef4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:21.972425 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82123c-82cd-4400-b796-ac143e14ef4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:21.972867 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82123c-82cd-4400-b796-ac143e14ef4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:21.973203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82123c-82cd-4400-b796-ac143e14ef4f admin admin] PATH_INFO: `/v3/projects/3e9e7d6ad24542ccb6ee81fea23ecf21` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:21.975653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82123c-82cd-4400-b796-ac143e14ef4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e9e7d6ad24542ccb6ee81fea23ecf21)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:21.977895 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82123c-82cd-4400-b796-ac143e14ef4f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:22.050723 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-43e0cfe9-1d1a-4c33-b2d5-759a7dfc4e8b None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:22.054489 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-43e0cfe9-1d1a-4c33-b2d5-759a7dfc4e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:22.079104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 907/1830] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:22.090942 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812391a6-19f7-46d7-af29-8cdfb0e33731 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:22.091364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812391a6-19f7-46d7-af29-8cdfb0e33731 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:22.091774 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812391a6-19f7-46d7-af29-8cdfb0e33731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:22.137214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-812391a6-19f7-46d7-af29-8cdfb0e33731 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:22.139481 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-812391a6-19f7-46d7-af29-8cdfb0e33731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:22.162740 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0b82123c-82cd-4400-b796-ac143e14ef4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e9e7d6ad24542ccb6ee81fea23ecf21', 'request_id': 'req-0b82123c-82cd-4400-b796-ac143e14ef4f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:22.182678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0b82123c-82cd-4400-b796-ac143e14ef4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e9e7d6ad24542ccb6ee81fea23ecf21', 'request_id': 'req-0b82123c-82cd-4400-b796-ac143e14ef4f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:22.203580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0b82123c-82cd-4400-b796-ac143e14ef4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e9e7d6ad24542ccb6ee81fea23ecf21', 'request_id': 'req-0b82123c-82cd-4400-b796-ac143e14ef4f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:22.228035 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0b82123c-82cd-4400-b796-ac143e14ef4f admin admin] The token cache is being invalidate because project 3e9e7d6ad24542ccb6ee81fea23ecf21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:22.228665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0b82123c-82cd-4400-b796-ac143e14ef4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:22.230217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0b82123c-82cd-4400-b796-ac143e14ef4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:22.233418 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 924/1831] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:21 2022] DELETE /identity/v3/projects/3e9e7d6ad24542ccb6ee81fea23ecf21 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:25:22.242366 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d422c-3160-482e-8918-383ab887ef2d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:22.243357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d422c-3160-482e-8918-383ab887ef2d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:22.244067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d422c-3160-482e-8918-383ab887ef2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:22.287356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 908/1832] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:22.292821 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812391a6-19f7-46d7-af29-8cdfb0e33731 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:22.294475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-356d422c-3160-482e-8918-383ab887ef2d None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:22.300775 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-356d422c-3160-482e-8918-383ab887ef2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:22.310325 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812391a6-19f7-46d7-af29-8cdfb0e33731 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:22.310638 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 925/1833] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:22.323344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f47e4f-2196-48f3-8390-486e03c04dff None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:22.323703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f47e4f-2196-48f3-8390-486e03c04dff None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:22.323947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f47e4f-2196-48f3-8390-486e03c04dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:22.325955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388bb81-43d4-4659-ac7c-f5488f60b370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:22.327659 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388bb81-43d4-4659-ac7c-f5488f60b370 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:22.330177 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388bb81-43d4-4659-ac7c-f5488f60b370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:22.331414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388bb81-43d4-4659-ac7c-f5488f60b370 admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:22.334153 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388bb81-43d4-4659-ac7c-f5488f60b370 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:25:22.334153 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388bb81-43d4-4659-ac7c-f5488f60b370 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:22.337750 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388bb81-43d4-4659-ac7c-f5488f60b370 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:22.358852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-00f47e4f-2196-48f3-8390-486e03c04dff None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:22.361512 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-00f47e4f-2196-48f3-8390-486e03c04dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:22.372955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 909/1834] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:25:22.380943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 926/1835] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:22.382283 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388bb81-43d4-4659-ac7c-f5488f60b370 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:22.386109 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f47e4f-2196-48f3-8390-486e03c04dff None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:22.403006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f47e4f-2196-48f3-8390-486e03c04dff None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:22.406184 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388bb81-43d4-4659-ac7c-f5488f60b370 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:22.411317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d82bfb-cf04-4a37-bea0-41f9ef7cd91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:22.412264 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d82bfb-cf04-4a37-bea0-41f9ef7cd91b admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:22.412914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d82bfb-cf04-4a37-bea0-41f9ef7cd91b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:22.413459 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d82bfb-cf04-4a37-bea0-41f9ef7cd91b admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:22.414328 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d82bfb-cf04-4a37-bea0-41f9ef7cd91b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:25:22.414983 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d82bfb-cf04-4a37-bea0-41f9ef7cd91b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:22.421724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5b978-8f9f-4c3d-b6e0-06b3acd80ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:22.423008 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5b978-8f9f-4c3d-b6e0-06b3acd80ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:22.423579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5b978-8f9f-4c3d-b6e0-06b3acd80ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:22.425019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5b978-8f9f-4c3d-b6e0-06b3acd80ac7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:22.425552 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d82bfb-cf04-4a37-bea0-41f9ef7cd91b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:22.426222 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5b978-8f9f-4c3d-b6e0-06b3acd80ac7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:22.431466 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5b978-8f9f-4c3d-b6e0-06b3acd80ac7 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:22.448952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 927/1836] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:25:22.455970 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d82bfb-cf04-4a37-bea0-41f9ef7cd91b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:22.486246 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d82bfb-cf04-4a37-bea0-41f9ef7cd91b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:22.495242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f747eb-63b2-47a6-b66f-5dd887ddffab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:22.496098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f747eb-63b2-47a6-b66f-5dd887ddffab admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:22.496567 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f747eb-63b2-47a6-b66f-5dd887ddffab admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:22.497324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f747eb-63b2-47a6-b66f-5dd887ddffab admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:22.498561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f747eb-63b2-47a6-b66f-5dd887ddffab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:22.504128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f747eb-63b2-47a6-b66f-5dd887ddffab admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:22.510105 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 910/1837] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:22 2022] POST /identity/v3/projects => generated 389 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:22.521840 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5b978-8f9f-4c3d-b6e0-06b3acd80ac7 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:22.550638 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 928/1838] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:22 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:22.561219 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f747eb-63b2-47a6-b66f-5dd887ddffab admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:22.642528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:22.692322 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5b978-8f9f-4c3d-b6e0-06b3acd80ac7 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:22.701940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9564ae-89de-4f0d-9973-43f80250eefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:22.703018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:22.703274 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9564ae-89de-4f0d-9973-43f80250eefd admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:22.704767 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9564ae-89de-4f0d-9973-43f80250eefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:22.705239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9564ae-89de-4f0d-9973-43f80250eefd admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:22.706695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9564ae-89de-4f0d-9973-43f80250eefd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:22.712551 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9564ae-89de-4f0d-9973-43f80250eefd admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:22.719665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f747eb-63b2-47a6-b66f-5dd887ddffab admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:22.726552 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f820779-53b7-48dc-8ea7-3f89cc7ffb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:22.727287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f820779-53b7-48dc-8ea7-3f89cc7ffb17 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:22.727681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f820779-53b7-48dc-8ea7-3f89cc7ffb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:22.728026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f820779-53b7-48dc-8ea7-3f89cc7ffb17 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:22.728772 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f820779-53b7-48dc-8ea7-3f89cc7ffb17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:22.732163 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f820779-53b7-48dc-8ea7-3f89cc7ffb17 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:22.765265 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 911/1839] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:22 2022] POST /identity/v3/users => generated 295 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:25:22.765265 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9564ae-89de-4f0d-9973-43f80250eefd admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:22.766273 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0e9564ae-89de-4f0d-9973-43f80250eefd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:22.776614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 929/1840] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:22 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:25:22.778369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f820779-53b7-48dc-8ea7-3f89cc7ffb17 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:22.795087 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f820779-53b7-48dc-8ea7-3f89cc7ffb17 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:22.797656 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9564ae-89de-4f0d-9973-43f80250eefd admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:22.803435 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f79d80-45d0-4296-b7db-40be7c23781b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:22.804083 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f79d80-45d0-4296-b7db-40be7c23781b admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:22.804358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f79d80-45d0-4296-b7db-40be7c23781b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:22.804640 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f79d80-45d0-4296-b7db-40be7c23781b admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:22.805317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f79d80-45d0-4296-b7db-40be7c23781b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:22.805986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1147056-629b-462f-9d6d-503e744c3b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:22.806853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1147056-629b-462f-9d6d-503e744c3b8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:22.807041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f79d80-45d0-4296-b7db-40be7c23781b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:22.807636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1147056-629b-462f-9d6d-503e744c3b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:22.808127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1147056-629b-462f-9d6d-503e744c3b8f admin admin] PATH_INFO: `/v3/projects/755bbed556204670891cdbb630076bd0` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:22.810720 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1147056-629b-462f-9d6d-503e744c3b8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=755bbed556204670891cdbb630076bd0)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:22.813027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1147056-629b-462f-9d6d-503e744c3b8f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:22.825162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 930/1841] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:22.827067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f79d80-45d0-4296-b7db-40be7c23781b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:22.845770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f79d80-45d0-4296-b7db-40be7c23781b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:22.857399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5defa785-c324-48a3-9eaa-b766dc66d482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:22.858359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defa785-c324-48a3-9eaa-b766dc66d482 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:22.860120 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defa785-c324-48a3-9eaa-b766dc66d482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:22.860754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defa785-c324-48a3-9eaa-b766dc66d482 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:22.861691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5defa785-c324-48a3-9eaa-b766dc66d482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:22.863625 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5defa785-c324-48a3-9eaa-b766dc66d482 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:22.883200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 931/1842] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:22.884501 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5defa785-c324-48a3-9eaa-b766dc66d482 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:22.900377 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5defa785-c324-48a3-9eaa-b766dc66d482 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:22.918023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d1147056-629b-462f-9d6d-503e744c3b8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '755bbed556204670891cdbb630076bd0', 'request_id': 'req-d1147056-629b-462f-9d6d-503e744c3b8f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:22.932138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d1147056-629b-462f-9d6d-503e744c3b8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '755bbed556204670891cdbb630076bd0', 'request_id': 'req-d1147056-629b-462f-9d6d-503e744c3b8f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:22.946988 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d1147056-629b-462f-9d6d-503e744c3b8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '755bbed556204670891cdbb630076bd0', 'request_id': 'req-d1147056-629b-462f-9d6d-503e744c3b8f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:22.970405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d1147056-629b-462f-9d6d-503e744c3b8f admin admin] The token cache is being invalidate because project 755bbed556204670891cdbb630076bd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:22.970741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d1147056-629b-462f-9d6d-503e744c3b8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:22.971460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d1147056-629b-462f-9d6d-503e744c3b8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:22.973604 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 912/1843] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:22 2022] DELETE /identity/v3/projects/755bbed556204670891cdbb630076bd0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:22.974451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1147056-629b-462f-9d6d-503e744c3b8f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:22.976288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d1147056-629b-462f-9d6d-503e744c3b8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:23.024853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83876585-296c-48e4-8dd7-e27e854f67b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:23.025979 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83876585-296c-48e4-8dd7-e27e854f67b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:23.027393 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83876585-296c-48e4-8dd7-e27e854f67b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:23.027393 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83876585-296c-48e4-8dd7-e27e854f67b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:23.028695 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83876585-296c-48e4-8dd7-e27e854f67b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:23.032089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83876585-296c-48e4-8dd7-e27e854f67b7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:23.053763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 932/1844] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:23.054843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff976f88-3957-47bf-b931-a619518ae555 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:23.055302 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff976f88-3957-47bf-b931-a619518ae555 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:23.055671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff976f88-3957-47bf-b931-a619518ae555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:23.065734 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1147056-629b-462f-9d6d-503e744c3b8f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:23.071065 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5044e2f-99d3-4ea4-a357-706422b2e50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:23.071702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5044e2f-99d3-4ea4-a357-706422b2e50c admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:23.071984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5044e2f-99d3-4ea4-a357-706422b2e50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:23.072263 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5044e2f-99d3-4ea4-a357-706422b2e50c admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:23.072923 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5044e2f-99d3-4ea4-a357-706422b2e50c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:23.074597 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5044e2f-99d3-4ea4-a357-706422b2e50c admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:23.087715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 913/1845] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:23.088311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5044e2f-99d3-4ea4-a357-706422b2e50c admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:23.090117 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f5044e2f-99d3-4ea4-a357-706422b2e50c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:23.092988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ff976f88-3957-47bf-b931-a619518ae555 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:23.094961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ff976f88-3957-47bf-b931-a619518ae555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:23.105516 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5044e2f-99d3-4ea4-a357-706422b2e50c admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:23.110113 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 933/1846] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:23.113239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bab41b-9302-4557-bf6d-acfe40bb4538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:23.114118 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bab41b-9302-4557-bf6d-acfe40bb4538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:23.114506 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bab41b-9302-4557-bf6d-acfe40bb4538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:23.114890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bab41b-9302-4557-bf6d-acfe40bb4538 admin admin] PATH_INFO: `/v3/projects/9079dcaf74ec48778f67c5f0757235e7/users/e00391c84f1f46418ce9930426c3d1d0/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:23.116364 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff976f88-3957-47bf-b931-a619518ae555 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:23.134559 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bab41b-9302-4557-bf6d-acfe40bb4538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9079dcaf74ec48778f67c5f0757235e7, user_id=e00391c84f1f46418ce9930426c3d1d0, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:23.137518 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bab41b-9302-4557-bf6d-acfe40bb4538 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:23.138911 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff976f88-3957-47bf-b931-a619518ae555 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:23.147954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6e7fa-4709-44a0-9b48-5a01a973e266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:23.149541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6e7fa-4709-44a0-9b48-5a01a973e266 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:23.149765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6e7fa-4709-44a0-9b48-5a01a973e266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:23.150272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6e7fa-4709-44a0-9b48-5a01a973e266 admin admin] PATH_INFO: `/v3/projects/8f9711b791b546e7b5992d32cb8fcefc/users/e969a9f18e7d45d098808f3e26ffde7a/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:23.161179 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 914/1847] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:23 2022] PUT /identity/v3/projects/9079dcaf74ec48778f67c5f0757235e7/users/e00391c84f1f46418ce9930426c3d1d0/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:23.167688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4d348-7b42-46a2-b71f-04bee18b27fa None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:23.168314 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4d348-7b42-46a2-b71f-04bee18b27fa None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:23.169513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4d348-7b42-46a2-b71f-04bee18b27fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:23.181090 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6e7fa-4709-44a0-9b48-5a01a973e266 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9711b791b546e7b5992d32cb8fcefc, user_id=e969a9f18e7d45d098808f3e26ffde7a, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:23.187090 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6e7fa-4709-44a0-9b48-5a01a973e266 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:23.198560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-07e4d348-7b42-46a2-b71f-04bee18b27fa None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:23.201032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-07e4d348-7b42-46a2-b71f-04bee18b27fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:23.217172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 934/1848] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:23 2022] PUT /identity/v3/projects/8f9711b791b546e7b5992d32cb8fcefc/users/e969a9f18e7d45d098808f3e26ffde7a/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:23.217172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107c653-d62c-455f-8f23-73f194d65ba9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:23.217172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107c653-d62c-455f-8f23-73f194d65ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:23.217172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107c653-d62c-455f-8f23-73f194d65ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:23.245646 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:23.284170 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0107c653-d62c-455f-8f23-73f194d65ba9 None None] MFA Rules not processed for user `e00391c84f1f46418ce9930426c3d1d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:23.286639 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0107c653-d62c-455f-8f23-73f194d65ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:23.316069 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 915/1849] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:23.329566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a323d53-844d-44ab-8769-6e3260795db6 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:23.329566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a323d53-844d-44ab-8769-6e3260795db6 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:23.329566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a323d53-844d-44ab-8769-6e3260795db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:23.392833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5a323d53-844d-44ab-8769-6e3260795db6 None None] MFA Rules not processed for user `e969a9f18e7d45d098808f3e26ffde7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:23.395092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5a323d53-844d-44ab-8769-6e3260795db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:23.401701 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 935/1850] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:23.404419 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107c653-d62c-455f-8f23-73f194d65ba9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:23.418743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107c653-d62c-455f-8f23-73f194d65ba9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:23.425160 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b807a2b-df0b-4df9-b46f-499928fa5785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:23.426230 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b807a2b-df0b-4df9-b46f-499928fa5785 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:23.426456 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b807a2b-df0b-4df9-b46f-499928fa5785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:23.426745 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b807a2b-df0b-4df9-b46f-499928fa5785 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:23.436539 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b807a2b-df0b-4df9-b46f-499928fa5785 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:25:23.436539 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b807a2b-df0b-4df9-b46f-499928fa5785 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:23.436539 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b807a2b-df0b-4df9-b46f-499928fa5785 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:23.454534 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 936/1851] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:25:23.456257 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b807a2b-df0b-4df9-b46f-499928fa5785 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:23.467479 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:23.473538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b807a2b-df0b-4df9-b46f-499928fa5785 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:23.480746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5eed5-b549-45dd-b24b-018af9c67c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:23.481846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5eed5-b549-45dd-b24b-018af9c67c65 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:23.482070 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5eed5-b549-45dd-b24b-018af9c67c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:23.482269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5eed5-b549-45dd-b24b-018af9c67c65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:23.482836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5eed5-b549-45dd-b24b-018af9c67c65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:23.485244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5eed5-b549-45dd-b24b-018af9c67c65 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:23.525900 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 937/1852] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:23 2022] POST /identity/v3/projects => generated 387 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:23.527116 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5eed5-b549-45dd-b24b-018af9c67c65 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:23.545985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 916/1853] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:23.545985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a323d53-844d-44ab-8769-6e3260795db6 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:23.571819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a323d53-844d-44ab-8769-6e3260795db6 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:23.709391 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07afa80-3ad1-43fe-b13c-fa3fad565d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:23.710398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07afa80-3ad1-43fe-b13c-fa3fad565d13 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:23.711000 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07afa80-3ad1-43fe-b13c-fa3fad565d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:23.711528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07afa80-3ad1-43fe-b13c-fa3fad565d13 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:23.713107 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07afa80-3ad1-43fe-b13c-fa3fad565d13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:23.717184 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07afa80-3ad1-43fe-b13c-fa3fad565d13 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:23.734669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5eed5-b549-45dd-b24b-018af9c67c65 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:23.750708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686c78b-12e9-491e-a0d5-df01c9b768be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:23.752028 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686c78b-12e9-491e-a0d5-df01c9b768be admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:23.752652 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686c78b-12e9-491e-a0d5-df01c9b768be admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:23.753346 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686c78b-12e9-491e-a0d5-df01c9b768be admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:23.754759 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686c78b-12e9-491e-a0d5-df01c9b768be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:23.763611 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686c78b-12e9-491e-a0d5-df01c9b768be admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:23.784743 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 917/1854] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:23 2022] POST /identity/v3/users => generated 300 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:23.787082 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07afa80-3ad1-43fe-b13c-fa3fad565d13 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:23.816719 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07afa80-3ad1-43fe-b13c-fa3fad565d13 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:23.826182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdba77d-a98e-4cfb-956a-9b3154811646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:23.827725 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdba77d-a98e-4cfb-956a-9b3154811646 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:23.827725 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdba77d-a98e-4cfb-956a-9b3154811646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:23.828225 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdba77d-a98e-4cfb-956a-9b3154811646 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:23.829398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdba77d-a98e-4cfb-956a-9b3154811646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:23.833480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdba77d-a98e-4cfb-956a-9b3154811646 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:23.850553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 938/1855] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:23 2022] POST /identity/v3/projects => generated 409 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:23.851840 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686c78b-12e9-491e-a0d5-df01c9b768be admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:23.913896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 918/1856] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:23 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:23.915399 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdba77d-a98e-4cfb-956a-9b3154811646 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:23.960089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686c78b-12e9-491e-a0d5-df01c9b768be admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:24.002316 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:24.056460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abce1a4-31d3-44f0-bc62-1d25a7976a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:24.057847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abce1a4-31d3-44f0-bc62-1d25a7976a11 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:24.058386 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abce1a4-31d3-44f0-bc62-1d25a7976a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:24.059056 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abce1a4-31d3-44f0-bc62-1d25a7976a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:24.060630 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abce1a4-31d3-44f0-bc62-1d25a7976a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:24.063275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abce1a4-31d3-44f0-bc62-1d25a7976a11 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:24.069154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdba77d-a98e-4cfb-956a-9b3154811646 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:24.077170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c7877-e719-4b09-96a3-8bf003b871ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:24.077980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c7877-e719-4b09-96a3-8bf003b871ca admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:24.078356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c7877-e719-4b09-96a3-8bf003b871ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:24.078676 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c7877-e719-4b09-96a3-8bf003b871ca admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:24.079440 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c7877-e719-4b09-96a3-8bf003b871ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:24.081463 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 939/1857] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:24.084975 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c7877-e719-4b09-96a3-8bf003b871ca admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:24.086897 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abce1a4-31d3-44f0-bc62-1d25a7976a11 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:24.099817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abce1a4-31d3-44f0-bc62-1d25a7976a11 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:24.108130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7c507-4092-4748-97fd-1c30698f5bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:24.109267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7c507-4092-4748-97fd-1c30698f5bce admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:24.109784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7c507-4092-4748-97fd-1c30698f5bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:24.110497 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7c507-4092-4748-97fd-1c30698f5bce admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:24.111842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7c507-4092-4748-97fd-1c30698f5bce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:24.115293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7c507-4092-4748-97fd-1c30698f5bce admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:24.122062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 919/1858] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:23 2022] POST /identity/v3/users => generated 305 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:24.123814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c7877-e719-4b09-96a3-8bf003b871ca admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:24.130064 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:24.140777 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c7877-e719-4b09-96a3-8bf003b871ca admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:24.152367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c7885-bdcc-4558-9a98-b4dc6b7779a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:24.154557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c7885-bdcc-4558-9a98-b4dc6b7779a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:24.155102 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c7885-bdcc-4558-9a98-b4dc6b7779a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:24.155589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c7885-bdcc-4558-9a98-b4dc6b7779a1 admin admin] PATH_INFO: `/v3/projects/5fa7e427405b4ef288ecf69874bc2488/users/46c9bc23d2a84a4e98972f07ccdc6d88/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:24.162223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 940/1859] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:24 2022] POST /identity/v3/users => generated 294 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:24.163572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7c507-4092-4748-97fd-1c30698f5bce admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:24.178793 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7c507-4092-4748-97fd-1c30698f5bce admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:24.187145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5a757-0a5c-4d75-8e0c-aa9fda86bfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:24.187715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c7885-bdcc-4558-9a98-b4dc6b7779a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa7e427405b4ef288ecf69874bc2488, user_id=46c9bc23d2a84a4e98972f07ccdc6d88, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:24.188640 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5a757-0a5c-4d75-8e0c-aa9fda86bfa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:24.189292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5a757-0a5c-4d75-8e0c-aa9fda86bfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:24.189810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5a757-0a5c-4d75-8e0c-aa9fda86bfa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:24.190681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5a757-0a5c-4d75-8e0c-aa9fda86bfa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:24.192395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c7885-bdcc-4558-9a98-b4dc6b7779a1 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:24.193096 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5a757-0a5c-4d75-8e0c-aa9fda86bfa0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:24.214150 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 941/1860] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:24.217144 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5a757-0a5c-4d75-8e0c-aa9fda86bfa0 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:24.229201 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 920/1861] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:24 2022] PUT /identity/v3/projects/5fa7e427405b4ef288ecf69874bc2488/users/46c9bc23d2a84a4e98972f07ccdc6d88/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:24.229785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c7885-bdcc-4558-9a98-b4dc6b7779a1 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:24.234935 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5a757-0a5c-4d75-8e0c-aa9fda86bfa0 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:24.318517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c7885-bdcc-4558-9a98-b4dc6b7779a1 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:24.319216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f0994-b799-408d-ad4d-bbb15f122127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:24.319858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f0994-b799-408d-ad4d-bbb15f122127 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:24.320554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f0994-b799-408d-ad4d-bbb15f122127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:24.320750 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f0994-b799-408d-ad4d-bbb15f122127 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:24.321559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f0994-b799-408d-ad4d-bbb15f122127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:24.323367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f0994-b799-408d-ad4d-bbb15f122127 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:24.323654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b973f-c1d4-4e6d-9f69-b605c5a6fd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:24.324293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b973f-c1d4-4e6d-9f69-b605c5a6fd91 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:24.324865 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b973f-c1d4-4e6d-9f69-b605c5a6fd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:24.325194 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b973f-c1d4-4e6d-9f69-b605c5a6fd91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:24.326145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b973f-c1d4-4e6d-9f69-b605c5a6fd91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:24.327758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b973f-c1d4-4e6d-9f69-b605c5a6fd91 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:24.338139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 942/1862] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:24.338139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f0994-b799-408d-ad4d-bbb15f122127 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:24.342059 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 921/1863] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:24.343151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b973f-c1d4-4e6d-9f69-b605c5a6fd91 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:24.349964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f0994-b799-408d-ad4d-bbb15f122127 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:24.357149 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b973f-c1d4-4e6d-9f69-b605c5a6fd91 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:24.358044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7033b6b-b82a-4805-b41f-18905df72a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:24.359237 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7033b6b-b82a-4805-b41f-18905df72a2a admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:24.359596 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7033b6b-b82a-4805-b41f-18905df72a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:24.359943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7033b6b-b82a-4805-b41f-18905df72a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:24.360794 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7033b6b-b82a-4805-b41f-18905df72a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:24.365433 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7033b6b-b82a-4805-b41f-18905df72a2a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:24.365589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf40290-1dfb-4267-aa87-d77a835fdcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:24.366385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf40290-1dfb-4267-aa87-d77a835fdcf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:24.366754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf40290-1dfb-4267-aa87-d77a835fdcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:24.367035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf40290-1dfb-4267-aa87-d77a835fdcf2 admin admin] PATH_INFO: `/v3/projects/4897097892c947b88f6b5a102271d168/users/8f1a59986b4b4b7eac930611556d803c/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:24.382311 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 943/1864] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:24.384963 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf40290-1dfb-4267-aa87-d77a835fdcf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4897097892c947b88f6b5a102271d168, user_id=8f1a59986b4b4b7eac930611556d803c, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:24.386885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7033b6b-b82a-4805-b41f-18905df72a2a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:24.388565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf40290-1dfb-4267-aa87-d77a835fdcf2 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:24.401927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7033b6b-b82a-4805-b41f-18905df72a2a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:24.409511 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7f803-ca1a-4d56-a056-b5ab024f1fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:24.412259 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7f803-ca1a-4d56-a056-b5ab024f1fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:24.412639 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7f803-ca1a-4d56-a056-b5ab024f1fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:24.413010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7f803-ca1a-4d56-a056-b5ab024f1fb7 admin admin] PATH_INFO: `/v3/projects/b28ed2601de241e2a3ce3d766032e962/users/18d73fd6ba0441289604e600edb47a4c/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:24.426861 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 922/1865] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:24 2022] PUT /identity/v3/projects/4897097892c947b88f6b5a102271d168/users/8f1a59986b4b4b7eac930611556d803c/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:24.428962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf40290-1dfb-4267-aa87-d77a835fdcf2 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:24.445015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7f803-ca1a-4d56-a056-b5ab024f1fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b28ed2601de241e2a3ce3d766032e962, user_id=18d73fd6ba0441289604e600edb47a4c, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:24.448089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7f803-ca1a-4d56-a056-b5ab024f1fb7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:24.481599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 944/1866] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:24 2022] PUT /identity/v3/projects/b28ed2601de241e2a3ce3d766032e962/users/18d73fd6ba0441289604e600edb47a4c/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:24.481599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7f803-ca1a-4d56-a056-b5ab024f1fb7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:24.563214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf40290-1dfb-4267-aa87-d77a835fdcf2 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:24.582870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:24.595389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7f803-ca1a-4d56-a056-b5ab024f1fb7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:24.602660 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43342736-521e-431a-922f-c7ccc47a2875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:24.603404 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43342736-521e-431a-922f-c7ccc47a2875 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:24.603761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43342736-521e-431a-922f-c7ccc47a2875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:24.604124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43342736-521e-431a-922f-c7ccc47a2875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:24.605004 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43342736-521e-431a-922f-c7ccc47a2875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:24.608021 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43342736-521e-431a-922f-c7ccc47a2875 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:24.629521 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 945/1867] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:24.631389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94441abf-a9b0-406d-a6cb-551980823ccb None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:24.631872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94441abf-a9b0-406d-a6cb-551980823ccb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:24.632316 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94441abf-a9b0-406d-a6cb-551980823ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:24.676381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6abde-8add-451a-8ad7-27355b197bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:24.678633 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6abde-8add-451a-8ad7-27355b197bc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:24.680399 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6abde-8add-451a-8ad7-27355b197bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:24.681078 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6abde-8add-451a-8ad7-27355b197bc2 admin admin] PATH_INFO: `/v3/projects/5fa7e427405b4ef288ecf69874bc2488/users/46c9bc23d2a84a4e98972f07ccdc6d88/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:24.684890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6abde-8add-451a-8ad7-27355b197bc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa7e427405b4ef288ecf69874bc2488, user_id=46c9bc23d2a84a4e98972f07ccdc6d88, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:24.686120 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-94441abf-a9b0-406d-a6cb-551980823ccb None None] MFA Rules not processed for user `18d73fd6ba0441289604e600edb47a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:24.688692 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-94441abf-a9b0-406d-a6cb-551980823ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:24.692426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6abde-8add-451a-8ad7-27355b197bc2 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:24.716879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 923/1868] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:24 2022] PUT /identity/v3/projects/5fa7e427405b4ef288ecf69874bc2488/users/46c9bc23d2a84a4e98972f07ccdc6d88/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:24.718969 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6abde-8add-451a-8ad7-27355b197bc2 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:24.759885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:24.848060 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6abde-8add-451a-8ad7-27355b197bc2 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:24.859739 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 946/1869] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:24 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:24.859739 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc517bd6-cc80-4140-a60c-9c80c7cbb1ad None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:24.859739 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc517bd6-cc80-4140-a60c-9c80c7cbb1ad None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:24.859739 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc517bd6-cc80-4140-a60c-9c80c7cbb1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:24.864053 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb60d37-fdd5-432a-8ae5-969dfebf1411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:24.865195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb60d37-fdd5-432a-8ae5-969dfebf1411 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:24.865702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb60d37-fdd5-432a-8ae5-969dfebf1411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:24.866214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb60d37-fdd5-432a-8ae5-969dfebf1411 admin admin] PATH_INFO: `/v3/projects/4897097892c947b88f6b5a102271d168/users/8f1a59986b4b4b7eac930611556d803c/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:24.868564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb60d37-fdd5-432a-8ae5-969dfebf1411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4897097892c947b88f6b5a102271d168, user_id=8f1a59986b4b4b7eac930611556d803c, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:24.874885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb60d37-fdd5-432a-8ae5-969dfebf1411 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:24.895477 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 924/1870] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:24 2022] PUT /identity/v3/projects/4897097892c947b88f6b5a102271d168/users/8f1a59986b4b4b7eac930611556d803c/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:24.896690 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb60d37-fdd5-432a-8ae5-969dfebf1411 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:24.938839 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fc517bd6-cc80-4140-a60c-9c80c7cbb1ad None None] MFA Rules not processed for user `46c9bc23d2a84a4e98972f07ccdc6d88`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:24.941422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fc517bd6-cc80-4140-a60c-9c80c7cbb1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:25.062117 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb60d37-fdd5-432a-8ae5-969dfebf1411 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:25.083556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4b276-b973-4dd2-aed0-f21ce892e61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:25.084213 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4b276-b973-4dd2-aed0-f21ce892e61c admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:25.084661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4b276-b973-4dd2-aed0-f21ce892e61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:25.085509 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4b276-b973-4dd2-aed0-f21ce892e61c admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:25.086194 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4b276-b973-4dd2-aed0-f21ce892e61c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:25.090167 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4b276-b973-4dd2-aed0-f21ce892e61c admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:25.117386 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 947/1871] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:25.118213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3391d37-ac00-41a6-bd00-ce3fe9e40df9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:25.119053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3391d37-ac00-41a6-bd00-ce3fe9e40df9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:25.119334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3391d37-ac00-41a6-bd00-ce3fe9e40df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:25.156442 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 925/1872] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:24 2022] POST /identity/v3/projects => generated 407 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:25.157962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4b276-b973-4dd2-aed0-f21ce892e61c admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:25.163518 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4ef4b276-b973-4dd2-aed0-f21ce892e61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:25.221247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e3391d37-ac00-41a6-bd00-ce3fe9e40df9 None None] MFA Rules not processed for user `8f1a59986b4b4b7eac930611556d803c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:25.226110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e3391d37-ac00-41a6-bd00-ce3fe9e40df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:25.252003 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:25.289878 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4b276-b973-4dd2-aed0-f21ce892e61c admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:25.296345 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607eede-5b44-4768-8def-3ad01e1982dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:25.297276 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607eede-5b44-4768-8def-3ad01e1982dc service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:25.297701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607eede-5b44-4768-8def-3ad01e1982dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:25.298129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607eede-5b44-4768-8def-3ad01e1982dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:25.400308 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607eede-5b44-4768-8def-3ad01e1982dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:25.402087 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607eede-5b44-4768-8def-3ad01e1982dc service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:25.405762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 948/1873] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:25.405762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3391d37-ac00-41a6-bd00-ce3fe9e40df9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:25.419533 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 926/1874] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:25.420845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607eede-5b44-4768-8def-3ad01e1982dc service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:25.423005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-2607eede-5b44-4768-8def-3ad01e1982dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:25.439539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607eede-5b44-4768-8def-3ad01e1982dc service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:25.444098 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d86e5-06c5-4b4e-bca3-07d9401cd73a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:25.445044 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d86e5-06c5-4b4e-bca3-07d9401cd73a service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:25.445456 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d86e5-06c5-4b4e-bca3-07d9401cd73a service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:25.445833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d86e5-06c5-4b4e-bca3-07d9401cd73a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:25.484230 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:25.536967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3391d37-ac00-41a6-bd00-ce3fe9e40df9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:25.552196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f7d5b-f4fd-4f6c-8045-2a0b82142209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:25.552857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f7d5b-f4fd-4f6c-8045-2a0b82142209 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:25.553164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f7d5b-f4fd-4f6c-8045-2a0b82142209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:25.553544 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f7d5b-f4fd-4f6c-8045-2a0b82142209 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:25.554085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f7d5b-f4fd-4f6c-8045-2a0b82142209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:25.556101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f7d5b-f4fd-4f6c-8045-2a0b82142209 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:25.569526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d86e5-06c5-4b4e-bca3-07d9401cd73a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:25.574209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d86e5-06c5-4b4e-bca3-07d9401cd73a service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:25.610228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 927/1875] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:25.622535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 949/1876] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:25 2022] POST /identity/v3/users => generated 310 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:25.627859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d86e5-06c5-4b4e-bca3-07d9401cd73a service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:25.689338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d86e5-06c5-4b4e-bca3-07d9401cd73a service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:25.713631 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843f4a6-41f0-47c5-ae60-f6213e01b102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:25.714459 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843f4a6-41f0-47c5-ae60-f6213e01b102 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:25.714753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843f4a6-41f0-47c5-ae60-f6213e01b102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:25.717615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843f4a6-41f0-47c5-ae60-f6213e01b102 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:25.718577 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843f4a6-41f0-47c5-ae60-f6213e01b102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:25.722207 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843f4a6-41f0-47c5-ae60-f6213e01b102 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:25.763362 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 928/1877] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:25.764761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f7d5b-f4fd-4f6c-8045-2a0b82142209 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:25.817320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f7d5b-f4fd-4f6c-8045-2a0b82142209 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:25.831449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843f4a6-41f0-47c5-ae60-f6213e01b102 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:25.885153 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea9b5c-64d4-482d-a6ba-2d95b94fa3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:25.888564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea9b5c-64d4-482d-a6ba-2d95b94fa3b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:25.888564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea9b5c-64d4-482d-a6ba-2d95b94fa3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:25.902679 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea9b5c-64d4-482d-a6ba-2d95b94fa3b9 admin admin] PATH_INFO: `/v3/projects/89e3fa5077124564bf7478a134c808fa/users/a428854028574a59a38aad240050be48/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:25.907616 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843f4a6-41f0-47c5-ae60-f6213e01b102 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:25.938908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b203e95-b926-4b41-8533-3b6f38df11d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:25.943470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b203e95-b926-4b41-8533-3b6f38df11d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:25.948927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b203e95-b926-4b41-8533-3b6f38df11d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:25.952314 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b203e95-b926-4b41-8533-3b6f38df11d6 admin admin] PATH_INFO: `/v3/users/e00391c84f1f46418ce9930426c3d1d0` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:25.975982 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b203e95-b926-4b41-8533-3b6f38df11d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e00391c84f1f46418ce9930426c3d1d0)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:25.978252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b203e95-b926-4b41-8533-3b6f38df11d6 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:25.987597 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea9b5c-64d4-482d-a6ba-2d95b94fa3b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e3fa5077124564bf7478a134c808fa, user_id=a428854028574a59a38aad240050be48, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:25.997089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea9b5c-64d4-482d-a6ba-2d95b94fa3b9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:26.068790 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 950/1878] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:25 2022] PUT /identity/v3/projects/89e3fa5077124564bf7478a134c808fa/users/a428854028574a59a38aad240050be48/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:26.073093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea9b5c-64d4-482d-a6ba-2d95b94fa3b9 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:26.163291 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4b203e95-b926-4b41-8533-3b6f38df11d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e00391c84f1f46418ce9930426c3d1d0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:26.165019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4b203e95-b926-4b41-8533-3b6f38df11d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e00391c84f1f46418ce9930426c3d1d0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:26.182414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4b203e95-b926-4b41-8533-3b6f38df11d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e00391c84f1f46418ce9930426c3d1d0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:26.190924 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:26.227564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea9b5c-64d4-482d-a6ba-2d95b94fa3b9 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:26.233887 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4b203e95-b926-4b41-8533-3b6f38df11d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e00391c84f1f46418ce9930426c3d1d0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:26.237900 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5d8be-97b0-4f7b-bd33-d595d30b717a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:26.239541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5d8be-97b0-4f7b-bd33-d595d30b717a admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:26.240185 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5d8be-97b0-4f7b-bd33-d595d30b717a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:26.240762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5d8be-97b0-4f7b-bd33-d595d30b717a admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:26.242092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de5d8be-97b0-4f7b-bd33-d595d30b717a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:26.246023 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de5d8be-97b0-4f7b-bd33-d595d30b717a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:26.261850 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4b203e95-b926-4b41-8533-3b6f38df11d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e00391c84f1f46418ce9930426c3d1d0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:26.269399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 951/1879] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:25:26.274912 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5d8be-97b0-4f7b-bd33-d595d30b717a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:26.277332 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 929/1880] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:25 2022] DELETE /identity/v3/users/e00391c84f1f46418ce9930426c3d1d0 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:26.282770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704754f6-bfd3-4ee8-8951-560b31c6bfe6 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:26.283204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704754f6-bfd3-4ee8-8951-560b31c6bfe6 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:26.283480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704754f6-bfd3-4ee8-8951-560b31c6bfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:26.290626 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5de5d8be-97b0-4f7b-bd33-d595d30b717a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:26.320373 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-704754f6-bfd3-4ee8-8951-560b31c6bfe6 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:26.323205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-704754f6-bfd3-4ee8-8951-560b31c6bfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:26.334538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5d8be-97b0-4f7b-bd33-d595d30b717a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:26.508530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 930/1881] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:26.548670 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704754f6-bfd3-4ee8-8951-560b31c6bfe6 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:26.550217 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-704754f6-bfd3-4ee8-8951-560b31c6bfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:26.581692 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ba9c5-1c2d-40fe-b6a6-6a57d8fe9bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:26.588364 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ba9c5-1c2d-40fe-b6a6-6a57d8fe9bce admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:26.589346 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ba9c5-1c2d-40fe-b6a6-6a57d8fe9bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:26.590037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ba9c5-1c2d-40fe-b6a6-6a57d8fe9bce admin admin] PATH_INFO: `/v3/projects/89e3fa5077124564bf7478a134c808fa/users/a428854028574a59a38aad240050be48/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:26.597706 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ba9c5-1c2d-40fe-b6a6-6a57d8fe9bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e3fa5077124564bf7478a134c808fa, user_id=a428854028574a59a38aad240050be48, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:26.605637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ba9c5-1c2d-40fe-b6a6-6a57d8fe9bce admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:26.641907 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 952/1882] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:26 2022] PUT /identity/v3/projects/89e3fa5077124564bf7478a134c808fa/users/a428854028574a59a38aad240050be48/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:26.657356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094c09b-da9e-4c9e-a6a2-49d76fcc1662 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:26.657943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094c09b-da9e-4c9e-a6a2-49d76fcc1662 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:26.661382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094c09b-da9e-4c9e-a6a2-49d76fcc1662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:26.698241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704754f6-bfd3-4ee8-8951-560b31c6bfe6 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:26.740024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f094c09b-da9e-4c9e-a6a2-49d76fcc1662 None None] MFA Rules not processed for user `a428854028574a59a38aad240050be48`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:26.742126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f094c09b-da9e-4c9e-a6a2-49d76fcc1662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:26.814627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:26.873162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab02f8-3763-4b5e-a7c9-4f9a882004f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:26.874212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab02f8-3763-4b5e-a7c9-4f9a882004f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:26.874575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab02f8-3763-4b5e-a7c9-4f9a882004f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:26.874926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab02f8-3763-4b5e-a7c9-4f9a882004f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:26.915637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 953/1883] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:26 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:26.927940 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094c09b-da9e-4c9e-a6a2-49d76fcc1662 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:26.929322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f094c09b-da9e-4c9e-a6a2-49d76fcc1662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:26.979138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:27.024774 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab02f8-3763-4b5e-a7c9-4f9a882004f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:27.029607 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab02f8-3763-4b5e-a7c9-4f9a882004f5 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:27.036336 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094c09b-da9e-4c9e-a6a2-49d76fcc1662 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:27.041669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a7af6-f8f4-4b6c-8a59-06b9a74a79cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:27.042924 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a7af6-f8f4-4b6c-8a59-06b9a74a79cf service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:27.044056 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a7af6-f8f4-4b6c-8a59-06b9a74a79cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:27.044529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a7af6-f8f4-4b6c-8a59-06b9a74a79cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:27.057119 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 931/1884] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:25:27.064329 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a7af6-f8f4-4b6c-8a59-06b9a74a79cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:27.066084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a7af6-f8f4-4b6c-8a59-06b9a74a79cf service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:27.079983 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 954/1885] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:26 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:25:27.341930 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab02f8-3763-4b5e-a7c9-4f9a882004f5 service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:27.356634 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab02f8-3763-4b5e-a7c9-4f9a882004f5 service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:27.365579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a155124-e60c-41d6-989f-81c4c9e5d8e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:27.366720 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a155124-e60c-41d6-989f-81c4c9e5d8e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:27.367307 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a155124-e60c-41d6-989f-81c4c9e5d8e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:27.367889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a155124-e60c-41d6-989f-81c4c9e5d8e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:27.375694 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-1a155124-e60c-41d6-989f-81c4c9e5d8e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:27.500564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a155124-e60c-41d6-989f-81c4c9e5d8e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:27.503974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a155124-e60c-41d6-989f-81c4c9e5d8e8 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:27.522624 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 932/1886] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:27 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:27.619602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a7af6-f8f4-4b6c-8a59-06b9a74a79cf service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:27.625597 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-868a7af6-f8f4-4b6c-8a59-06b9a74a79cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:27.674622 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a7af6-f8f4-4b6c-8a59-06b9a74a79cf service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:27.701946 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa29dc6-3613-4b83-8072-12537be3ff5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:27.701946 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa29dc6-3613-4b83-8072-12537be3ff5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:27.701946 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa29dc6-3613-4b83-8072-12537be3ff5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:27.701946 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa29dc6-3613-4b83-8072-12537be3ff5a admin admin] PATH_INFO: `/v3/projects/9079dcaf74ec48778f67c5f0757235e7` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:27.702693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa29dc6-3613-4b83-8072-12537be3ff5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9079dcaf74ec48778f67c5f0757235e7)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:27.702693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa29dc6-3613-4b83-8072-12537be3ff5a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:27.822933 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6fa29dc6-3613-4b83-8072-12537be3ff5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9079dcaf74ec48778f67c5f0757235e7', 'request_id': 'req-6fa29dc6-3613-4b83-8072-12537be3ff5a'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:27.851931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6fa29dc6-3613-4b83-8072-12537be3ff5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9079dcaf74ec48778f67c5f0757235e7', 'request_id': 'req-6fa29dc6-3613-4b83-8072-12537be3ff5a'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:27.889678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6fa29dc6-3613-4b83-8072-12537be3ff5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9079dcaf74ec48778f67c5f0757235e7', 'request_id': 'req-6fa29dc6-3613-4b83-8072-12537be3ff5a'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:27.923441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6fa29dc6-3613-4b83-8072-12537be3ff5a admin admin] The token cache is being invalidate because project 9079dcaf74ec48778f67c5f0757235e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:27.925657 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6fa29dc6-3613-4b83-8072-12537be3ff5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:27.927795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6fa29dc6-3613-4b83-8072-12537be3ff5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:27.942295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a155124-e60c-41d6-989f-81c4c9e5d8e8 service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:27.943975 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 955/1887] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:27 2022] DELETE /identity/v3/projects/9079dcaf74ec48778f67c5f0757235e7 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:27.991009 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a155124-e60c-41d6-989f-81c4c9e5d8e8 service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:28.265045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b645b63-62e8-44dd-a20e-167195c26700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:28.265829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b645b63-62e8-44dd-a20e-167195c26700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:28.266481 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b645b63-62e8-44dd-a20e-167195c26700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:28.266920 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b645b63-62e8-44dd-a20e-167195c26700 admin admin] PATH_INFO: `/v3/users/46c9bc23d2a84a4e98972f07ccdc6d88` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:28.271164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b645b63-62e8-44dd-a20e-167195c26700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46c9bc23d2a84a4e98972f07ccdc6d88)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:28.283559 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b645b63-62e8-44dd-a20e-167195c26700 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:28.433445 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0b645b63-62e8-44dd-a20e-167195c26700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46c9bc23d2a84a4e98972f07ccdc6d88'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:28.434892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0b645b63-62e8-44dd-a20e-167195c26700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46c9bc23d2a84a4e98972f07ccdc6d88'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:28.451080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0b645b63-62e8-44dd-a20e-167195c26700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46c9bc23d2a84a4e98972f07ccdc6d88'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:28.482271 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0b645b63-62e8-44dd-a20e-167195c26700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46c9bc23d2a84a4e98972f07ccdc6d88'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:28.498840 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0b645b63-62e8-44dd-a20e-167195c26700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46c9bc23d2a84a4e98972f07ccdc6d88'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:28.512896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 933/1888] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:27 2022] DELETE /identity/v3/users/46c9bc23d2a84a4e98972f07ccdc6d88 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:29.097186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa29dc6-3613-4b83-8072-12537be3ff5a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:29.099459 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6fa29dc6-3613-4b83-8072-12537be3ff5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:29.115690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa29dc6-3613-4b83-8072-12537be3ff5a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:29.177382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:29.198663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b7d5b-21d8-420c-aeea-9d953a11a863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:29.200399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b7d5b-21d8-420c-aeea-9d953a11a863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:29.200874 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b7d5b-21d8-420c-aeea-9d953a11a863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:29.201226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b7d5b-21d8-420c-aeea-9d953a11a863 admin admin] PATH_INFO: `/v3/projects/5fa7e427405b4ef288ecf69874bc2488` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:29.202607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b7d5b-21d8-420c-aeea-9d953a11a863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fa7e427405b4ef288ecf69874bc2488)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:29.204957 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b7d5b-21d8-420c-aeea-9d953a11a863 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:29.266012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bc4b7d5b-21d8-420c-aeea-9d953a11a863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fa7e427405b4ef288ecf69874bc2488', 'request_id': 'req-bc4b7d5b-21d8-420c-aeea-9d953a11a863'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:29.279067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bc4b7d5b-21d8-420c-aeea-9d953a11a863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fa7e427405b4ef288ecf69874bc2488', 'request_id': 'req-bc4b7d5b-21d8-420c-aeea-9d953a11a863'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:29.286373 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b645b63-62e8-44dd-a20e-167195c26700 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:29.289476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0b645b63-62e8-44dd-a20e-167195c26700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:29.290387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bc4b7d5b-21d8-420c-aeea-9d953a11a863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fa7e427405b4ef288ecf69874bc2488', 'request_id': 'req-bc4b7d5b-21d8-420c-aeea-9d953a11a863'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:29.317234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bc4b7d5b-21d8-420c-aeea-9d953a11a863 admin admin] The token cache is being invalidate because project 5fa7e427405b4ef288ecf69874bc2488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:29.317723 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bc4b7d5b-21d8-420c-aeea-9d953a11a863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:29.318960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bc4b7d5b-21d8-420c-aeea-9d953a11a863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:29.321778 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 956/1889] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:29 2022] DELETE /identity/v3/projects/5fa7e427405b4ef288ecf69874bc2488 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:29.330446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374a8b8-1e59-48b4-a19b-c2ede0b2d6f4 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:29.330915 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374a8b8-1e59-48b4-a19b-c2ede0b2d6f4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:29.331378 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374a8b8-1e59-48b4-a19b-c2ede0b2d6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:29.334536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:29.344352 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b645b63-62e8-44dd-a20e-167195c26700 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:29.381366 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d374a8b8-1e59-48b4-a19b-c2ede0b2d6f4 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:29.382674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d374a8b8-1e59-48b4-a19b-c2ede0b2d6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:29.445728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb9952-13b9-44bc-bf16-60e2f78e3e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:29.445728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb9952-13b9-44bc-bf16-60e2f78e3e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:29.445728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb9952-13b9-44bc-bf16-60e2f78e3e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:29.445728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb9952-13b9-44bc-bf16-60e2f78e3e9c admin admin] PATH_INFO: `/v3/users/e969a9f18e7d45d098808f3e26ffde7a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:29.446336 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb9952-13b9-44bc-bf16-60e2f78e3e9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e969a9f18e7d45d098808f3e26ffde7a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:29.451570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb9952-13b9-44bc-bf16-60e2f78e3e9c admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:29.533872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 957/1890] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:29.536594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30288b6-1ff0-4d03-8126-82ebcaeac57c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:29.537191 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30288b6-1ff0-4d03-8126-82ebcaeac57c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:29.537676 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30288b6-1ff0-4d03-8126-82ebcaeac57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:29.585819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a30288b6-1ff0-4d03-8126-82ebcaeac57c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:29.590873 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a30288b6-1ff0-4d03-8126-82ebcaeac57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:29.607550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-abdb9952-13b9-44bc-bf16-60e2f78e3e9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e969a9f18e7d45d098808f3e26ffde7a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:29.612345 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 958/1891] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:29.613386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-abdb9952-13b9-44bc-bf16-60e2f78e3e9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e969a9f18e7d45d098808f3e26ffde7a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:29.619371 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30288b6-1ff0-4d03-8126-82ebcaeac57c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:29.650748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-abdb9952-13b9-44bc-bf16-60e2f78e3e9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e969a9f18e7d45d098808f3e26ffde7a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:29.655035 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30288b6-1ff0-4d03-8126-82ebcaeac57c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:29.677597 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4ac32-7f63-4015-800f-27b44801e2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:29.678354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4ac32-7f63-4015-800f-27b44801e2bc admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:29.679093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4ac32-7f63-4015-800f-27b44801e2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:29.679093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4ac32-7f63-4015-800f-27b44801e2bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:29.679779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4ac32-7f63-4015-800f-27b44801e2bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:25:29.681721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4ac32-7f63-4015-800f-27b44801e2bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:29.687903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4ac32-7f63-4015-800f-27b44801e2bc admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:29.720215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-abdb9952-13b9-44bc-bf16-60e2f78e3e9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e969a9f18e7d45d098808f3e26ffde7a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:29.731588 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 959/1892] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:25:29.733617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4ac32-7f63-4015-800f-27b44801e2bc admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:29.747866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-abdb9952-13b9-44bc-bf16-60e2f78e3e9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e969a9f18e7d45d098808f3e26ffde7a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:29.761666 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 934/1893] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:29 2022] DELETE /identity/v3/users/e969a9f18e7d45d098808f3e26ffde7a => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:29.764049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbde50-4871-4661-9a17-5d3cda0376c0 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:29.764343 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbde50-4871-4661-9a17-5d3cda0376c0 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:29.764614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbde50-4871-4661-9a17-5d3cda0376c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:29.801114 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-a8dbde50-4871-4661-9a17-5d3cda0376c0 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:29.804091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a8dbde50-4871-4661-9a17-5d3cda0376c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:29.837160 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4ac32-7f63-4015-800f-27b44801e2bc admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:30.003925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 935/1894] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:30.052951 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:30.078895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbde50-4871-4661-9a17-5d3cda0376c0 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:30.082398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a8dbde50-4871-4661-9a17-5d3cda0376c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:30.086337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0528b55-0dff-40cc-8d60-73c3e3c8b680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:30.087412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0528b55-0dff-40cc-8d60-73c3e3c8b680 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:30.087907 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0528b55-0dff-40cc-8d60-73c3e3c8b680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:30.088393 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0528b55-0dff-40cc-8d60-73c3e3c8b680 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:30.089940 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0528b55-0dff-40cc-8d60-73c3e3c8b680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:30.101919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0528b55-0dff-40cc-8d60-73c3e3c8b680 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:30.169996 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:30.197142 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 960/1895] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:29 2022] POST /identity/v3/projects => generated 377 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:30.202330 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0528b55-0dff-40cc-8d60-73c3e3c8b680 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:30.209088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a0528b55-0dff-40cc-8d60-73c3e3c8b680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:30.241219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbde50-4871-4661-9a17-5d3cda0376c0 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:30.339628 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89451cb7-8707-4f05-be7c-ccaee5805093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:30.340914 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89451cb7-8707-4f05-be7c-ccaee5805093 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:30.341243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89451cb7-8707-4f05-be7c-ccaee5805093 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:30.341426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89451cb7-8707-4f05-be7c-ccaee5805093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:30.368258 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0528b55-0dff-40cc-8d60-73c3e3c8b680 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:30.390382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a7f44-d1a8-4ab0-937d-ce19c8759497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:30.390382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a7f44-d1a8-4ab0-937d-ce19c8759497 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:30.390382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a7f44-d1a8-4ab0-937d-ce19c8759497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:30.390382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a7f44-d1a8-4ab0-937d-ce19c8759497 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:30.390725 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a7f44-d1a8-4ab0-937d-ce19c8759497 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:30.392373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a7f44-d1a8-4ab0-937d-ce19c8759497 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:30.432395 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 961/1896] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:30 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:30.438227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a7f44-d1a8-4ab0-937d-ce19c8759497 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:30.463746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a7f44-d1a8-4ab0-937d-ce19c8759497 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:30.479919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c707e-1fc8-4eec-b97c-f0d56338840a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:30.483592 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c707e-1fc8-4eec-b97c-f0d56338840a admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:30.484555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c707e-1fc8-4eec-b97c-f0d56338840a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:30.485413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c707e-1fc8-4eec-b97c-f0d56338840a admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:30.489802 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c707e-1fc8-4eec-b97c-f0d56338840a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:30.493401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c707e-1fc8-4eec-b97c-f0d56338840a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:30.503685 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89451cb7-8707-4f05-be7c-ccaee5805093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:30.505963 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89451cb7-8707-4f05-be7c-ccaee5805093 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:30.526849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 962/1897] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:30.528503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 936/1898] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:25:30.533851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c707e-1fc8-4eec-b97c-f0d56338840a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:30.561389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c707e-1fc8-4eec-b97c-f0d56338840a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:30.569000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ff53a-2561-4292-b3ac-5cbd040d0953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:30.570020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ff53a-2561-4292-b3ac-5cbd040d0953 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:30.570020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ff53a-2561-4292-b3ac-5cbd040d0953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:30.570292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ff53a-2561-4292-b3ac-5cbd040d0953 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:30.571026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ff53a-2561-4292-b3ac-5cbd040d0953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:30.573090 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ff53a-2561-4292-b3ac-5cbd040d0953 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:30.600811 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 963/1899] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:30.603422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89451cb7-8707-4f05-be7c-ccaee5805093 service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:30.627664 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89451cb7-8707-4f05-be7c-ccaee5805093 service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:30.638609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0b337-edd0-42b3-b9b7-2800d379e1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:30.640346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0b337-edd0-42b3-b9b7-2800d379e1a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:30.641026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0b337-edd0-42b3-b9b7-2800d379e1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:30.641686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0b337-edd0-42b3-b9b7-2800d379e1a0 admin admin] PATH_INFO: `/v3/projects/b78c581171934f10abfd65dfc1cf90a0/users/d5ca1497d079419c93adf44549750181/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:30.659187 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bc522-40b5-42e5-be92-2274ea469f98 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:30.659750 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bc522-40b5-42e5-be92-2274ea469f98 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:30.660400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bc522-40b5-42e5-be92-2274ea469f98 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:30.663267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 964/1900] 10.222.0.88 () {54 vars in 921 bytes} [Tue Aug 23 22:25:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:25:30.677779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0b337-edd0-42b3-b9b7-2800d379e1a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b78c581171934f10abfd65dfc1cf90a0, user_id=d5ca1497d079419c93adf44549750181, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:30.681473 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0b337-edd0-42b3-b9b7-2800d379e1a0 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:30.683633 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62113b2-cc8d-4ecc-95b5-af0079a22d27 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:30.685841 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62113b2-cc8d-4ecc-95b5-af0079a22d27 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:30.686721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62113b2-cc8d-4ecc-95b5-af0079a22d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:30.706887 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 937/1901] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:30 2022] PUT /identity/v3/projects/b78c581171934f10abfd65dfc1cf90a0/users/d5ca1497d079419c93adf44549750181/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:30.719325 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42188f80-fe97-431d-b3a0-a2642c9a410c None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:30.719609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42188f80-fe97-431d-b3a0-a2642c9a410c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:30.719876 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42188f80-fe97-431d-b3a0-a2642c9a410c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:30.751667 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d62113b2-cc8d-4ecc-95b5-af0079a22d27 None None] MFA Rules not processed for user `e3517b3b077a4a1fa76163657b46930c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:30.755424 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d62113b2-cc8d-4ecc-95b5-af0079a22d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:30.786490 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-42188f80-fe97-431d-b3a0-a2642c9a410c None None] MFA Rules not processed for user `d5ca1497d079419c93adf44549750181`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:30.787956 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-42188f80-fe97-431d-b3a0-a2642c9a410c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:30.951308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 965/1902] 10.222.0.88 () {58 vars in 1018 bytes} [Tue Aug 23 22:25:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:30.964859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:30.981937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 938/1903] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:30.986675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62113b2-cc8d-4ecc-95b5-af0079a22d27 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:31.083948 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42188f80-fe97-431d-b3a0-a2642c9a410c None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:31.085951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-42188f80-fe97-431d-b3a0-a2642c9a410c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:31.159550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:31.195065 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62113b2-cc8d-4ecc-95b5-af0079a22d27 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:31.211151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3598499-9725-4961-8c14-b195f2d76d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:31.213459 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3598499-9725-4961-8c14-b195f2d76d21 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:31.214629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3598499-9725-4961-8c14-b195f2d76d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:31.216302 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3598499-9725-4961-8c14-b195f2d76d21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:31.217308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3598499-9725-4961-8c14-b195f2d76d21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:31.229927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3598499-9725-4961-8c14-b195f2d76d21 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:31.332656 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 966/1904] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:30 2022] POST /identity/v3/projects => generated 379 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:31.337905 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3598499-9725-4961-8c14-b195f2d76d21 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:31.366181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42188f80-fe97-431d-b3a0-a2642c9a410c None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:31.503460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:31.504991 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:31.505681 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:31.506378 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029 admin admin] PATH_INFO: `/v3/projects/8f9711b791b546e7b5992d32cb8fcefc` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:31.508732 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f9711b791b546e7b5992d32cb8fcefc)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:31.514711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:31.525759 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3598499-9725-4961-8c14-b195f2d76d21 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:31.533698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13dfac-5c6e-46ae-bd76-f15b9c4869c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:31.534608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13dfac-5c6e-46ae-bd76-f15b9c4869c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:31.534888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13dfac-5c6e-46ae-bd76-f15b9c4869c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:31.535166 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13dfac-5c6e-46ae-bd76-f15b9c4869c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:31.535972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13dfac-5c6e-46ae-bd76-f15b9c4869c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:31.538345 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13dfac-5c6e-46ae-bd76-f15b9c4869c9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:31.579539 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 967/1905] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:31 2022] POST /identity/v3/users => generated 290 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:25:31.581240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13dfac-5c6e-46ae-bd76-f15b9c4869c9 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:31.584712 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7e13dfac-5c6e-46ae-bd76-f15b9c4869c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:31.610517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f9711b791b546e7b5992d32cb8fcefc', 'request_id': 'req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:31.626332 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f9711b791b546e7b5992d32cb8fcefc', 'request_id': 'req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:31.647777 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f9711b791b546e7b5992d32cb8fcefc', 'request_id': 'req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:31.700817 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029 admin admin] The token cache is being invalidate because project 8f9711b791b546e7b5992d32cb8fcefc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:31.700817 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:31.701550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:31.703160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 939/1906] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:31 2022] DELETE /identity/v3/projects/8f9711b791b546e7b5992d32cb8fcefc => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:25:31.704025 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:31.705505 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:31.742535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13dfac-5c6e-46ae-bd76-f15b9c4869c9 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:31.764435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:31.820083 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:31.830558 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05e681-09fd-4b92-b8dd-b91e6f5ba029 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:31.844879 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828bf8b-94c1-4ade-bc9c-db88e2a8aec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:31.845139 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c3cfa-31c1-46fa-9390-8bb721e75bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:31.845682 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c3cfa-31c1-46fa-9390-8bb721e75bad admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:31.846020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c3cfa-31c1-46fa-9390-8bb721e75bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:31.846247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828bf8b-94c1-4ade-bc9c-db88e2a8aec3 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:31.846478 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c3cfa-31c1-46fa-9390-8bb721e75bad admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:31.846994 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828bf8b-94c1-4ade-bc9c-db88e2a8aec3 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:31.847317 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c3cfa-31c1-46fa-9390-8bb721e75bad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:31.847598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828bf8b-94c1-4ade-bc9c-db88e2a8aec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:31.849351 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c3cfa-31c1-46fa-9390-8bb721e75bad admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:31.850840 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3828bf8b-94c1-4ade-bc9c-db88e2a8aec3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:31.862603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 940/1907] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:31.864726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c3cfa-31c1-46fa-9390-8bb721e75bad admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:31.869701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-799c3cfa-31c1-46fa-9390-8bb721e75bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:31.873516 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3828bf8b-94c1-4ade-bc9c-db88e2a8aec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:31.875479 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3828bf8b-94c1-4ade-bc9c-db88e2a8aec3 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:31.884753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c3cfa-31c1-46fa-9390-8bb721e75bad admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:31.891488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 968/1908] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:31.891855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794927a-ba71-4edc-afe3-095b1225f826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:31.892358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828bf8b-94c1-4ade-bc9c-db88e2a8aec3 service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:31.892537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794927a-ba71-4edc-afe3-095b1225f826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:31.892803 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794927a-ba71-4edc-afe3-095b1225f826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:31.893181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794927a-ba71-4edc-afe3-095b1225f826 admin admin] PATH_INFO: `/v3/users/8f1a59986b4b4b7eac930611556d803c` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:31.894445 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794927a-ba71-4edc-afe3-095b1225f826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f1a59986b4b4b7eac930611556d803c)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:31.896383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794927a-ba71-4edc-afe3-095b1225f826 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:31.904451 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828bf8b-94c1-4ade-bc9c-db88e2a8aec3 service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:31.910924 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41d9de-f085-469c-9649-81f7bc5ab4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:31.911745 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41d9de-f085-469c-9649-81f7bc5ab4dc admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:31.912025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41d9de-f085-469c-9649-81f7bc5ab4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:31.912323 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41d9de-f085-469c-9649-81f7bc5ab4dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:31.912940 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41d9de-f085-469c-9649-81f7bc5ab4dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:31.914581 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41d9de-f085-469c-9649-81f7bc5ab4dc admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:31.947807 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 969/1909] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:31.953522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41d9de-f085-469c-9649-81f7bc5ab4dc admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:31.970339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41d9de-f085-469c-9649-81f7bc5ab4dc admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:31.980112 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bfd42-652d-40c2-bbbc-96be1e96804c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:31.982139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bfd42-652d-40c2-bbbc-96be1e96804c admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:31.982564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bfd42-652d-40c2-bbbc-96be1e96804c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:31.983073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bfd42-652d-40c2-bbbc-96be1e96804c admin admin] PATH_INFO: `/v3/projects/dbfc2abbcadc438da6c154f2abaecae6/users/fceab4f68b4c471290d95aa000f11474/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:31.983761 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d794927a-ba71-4edc-afe3-095b1225f826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f1a59986b4b4b7eac930611556d803c'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:31.984591 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d794927a-ba71-4edc-afe3-095b1225f826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f1a59986b4b4b7eac930611556d803c'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:31.999409 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d794927a-ba71-4edc-afe3-095b1225f826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f1a59986b4b4b7eac930611556d803c'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:32.010012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bfd42-652d-40c2-bbbc-96be1e96804c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbfc2abbcadc438da6c154f2abaecae6, user_id=fceab4f68b4c471290d95aa000f11474, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:32.013744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bfd42-652d-40c2-bbbc-96be1e96804c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:32.020757 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d794927a-ba71-4edc-afe3-095b1225f826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f1a59986b4b4b7eac930611556d803c'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:32.030254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 970/1910] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:31 2022] PUT /identity/v3/projects/dbfc2abbcadc438da6c154f2abaecae6/users/fceab4f68b4c471290d95aa000f11474/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:32.035610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d794927a-ba71-4edc-afe3-095b1225f826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f1a59986b4b4b7eac930611556d803c'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:32.042381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646f2b0-846d-4a30-8780-5fa938e6eedd None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:32.042665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646f2b0-846d-4a30-8780-5fa938e6eedd None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:32.042941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646f2b0-846d-4a30-8780-5fa938e6eedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:32.048511 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 941/1911] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:31 2022] DELETE /identity/v3/users/8f1a59986b4b4b7eac930611556d803c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:32.095017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-1646f2b0-846d-4a30-8780-5fa938e6eedd None None] MFA Rules not processed for user `fceab4f68b4c471290d95aa000f11474`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:32.097133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1646f2b0-846d-4a30-8780-5fa938e6eedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:32.190030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 971/1912] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:32.197421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794927a-ba71-4edc-afe3-095b1225f826 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:32.200414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d794927a-ba71-4edc-afe3-095b1225f826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:32.297580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794927a-ba71-4edc-afe3-095b1225f826 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:32.302409 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf284a39-fd9d-45d7-b30b-e9b522c01536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:32.303041 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf284a39-fd9d-45d7-b30b-e9b522c01536 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:32.303317 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf284a39-fd9d-45d7-b30b-e9b522c01536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:32.303586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf284a39-fd9d-45d7-b30b-e9b522c01536 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:32.304226 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf284a39-fd9d-45d7-b30b-e9b522c01536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:32.306187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf284a39-fd9d-45d7-b30b-e9b522c01536 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:32.346473 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 942/1913] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:32 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:32.347670 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646f2b0-846d-4a30-8780-5fa938e6eedd None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:32.361213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646f2b0-846d-4a30-8780-5fa938e6eedd None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:32.451559 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf284a39-fd9d-45d7-b30b-e9b522c01536 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:32.454166 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-bf284a39-fd9d-45d7-b30b-e9b522c01536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:32.471924 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a51a0a-f3c8-469c-9fb5-7cbb087fe343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:32.473426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a51a0a-f3c8-469c-9fb5-7cbb087fe343 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:32.474238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a51a0a-f3c8-469c-9fb5-7cbb087fe343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:32.474834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a51a0a-f3c8-469c-9fb5-7cbb087fe343 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:32.475866 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a51a0a-f3c8-469c-9fb5-7cbb087fe343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:32.478664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a51a0a-f3c8-469c-9fb5-7cbb087fe343 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:32.521348 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:32.539743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 972/1914] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:32 2022] POST /identity/v3/users => generated 296 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:32.546360 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a51a0a-f3c8-469c-9fb5-7cbb087fe343 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:32.576810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a51a0a-f3c8-469c-9fb5-7cbb087fe343 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:32.588278 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0b424-81cd-4a04-ba74-2e221338a293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:32.589340 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0b424-81cd-4a04-ba74-2e221338a293 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:32.590038 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0b424-81cd-4a04-ba74-2e221338a293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:32.590615 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0b424-81cd-4a04-ba74-2e221338a293 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:32.593628 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0b424-81cd-4a04-ba74-2e221338a293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:32.596616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0b424-81cd-4a04-ba74-2e221338a293 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:32.616549 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf284a39-fd9d-45d7-b30b-e9b522c01536 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:32.620232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 973/1915] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:32.625797 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0b424-81cd-4a04-ba74-2e221338a293 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:32.629416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc398a-f779-4325-bc25-37cbe58279a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:32.630283 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc398a-f779-4325-bc25-37cbe58279a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:32.630552 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc398a-f779-4325-bc25-37cbe58279a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:32.630839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc398a-f779-4325-bc25-37cbe58279a2 admin admin] PATH_INFO: `/v3/projects/4897097892c947b88f6b5a102271d168` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:32.632133 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc398a-f779-4325-bc25-37cbe58279a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4897097892c947b88f6b5a102271d168)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:32.634266 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc398a-f779-4325-bc25-37cbe58279a2 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:32.645287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0b424-81cd-4a04-ba74-2e221338a293 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:32.656823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920aa4d8-975a-4f65-981e-6d566a182f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:32.657701 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920aa4d8-975a-4f65-981e-6d566a182f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:32.657982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920aa4d8-975a-4f65-981e-6d566a182f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:32.658565 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920aa4d8-975a-4f65-981e-6d566a182f82 admin admin] PATH_INFO: `/v3/projects/7ec466531ece4564be190ad94ad765b5/users/1aa92e05305b45ae93a58016a235e860/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:32.674177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:32.680016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920aa4d8-975a-4f65-981e-6d566a182f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec466531ece4564be190ad94ad765b5, user_id=1aa92e05305b45ae93a58016a235e860, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:32.683195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920aa4d8-975a-4f65-981e-6d566a182f82 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:32.700435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-57fc398a-f779-4325-bc25-37cbe58279a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4897097892c947b88f6b5a102271d168', 'request_id': 'req-57fc398a-f779-4325-bc25-37cbe58279a2'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:32.707611 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 974/1916] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:32 2022] PUT /identity/v3/projects/7ec466531ece4564be190ad94ad765b5/users/1aa92e05305b45ae93a58016a235e860/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:32.710904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-57fc398a-f779-4325-bc25-37cbe58279a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4897097892c947b88f6b5a102271d168', 'request_id': 'req-57fc398a-f779-4325-bc25-37cbe58279a2'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:32.712770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920aa4d8-975a-4f65-981e-6d566a182f82 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:32.718002 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-57fc398a-f779-4325-bc25-37cbe58279a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4897097892c947b88f6b5a102271d168', 'request_id': 'req-57fc398a-f779-4325-bc25-37cbe58279a2'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:32.740833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-57fc398a-f779-4325-bc25-37cbe58279a2 admin admin] The token cache is being invalidate because project 4897097892c947b88f6b5a102271d168 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:32.741379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-57fc398a-f779-4325-bc25-37cbe58279a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:32.742513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-57fc398a-f779-4325-bc25-37cbe58279a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:32.744983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 943/1917] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:32 2022] DELETE /identity/v3/projects/4897097892c947b88f6b5a102271d168 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:32.751896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9456f9f-8c70-4b0a-aa1d-24ba5920797c None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:32.752518 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9456f9f-8c70-4b0a-aa1d-24ba5920797c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:32.753131 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9456f9f-8c70-4b0a-aa1d-24ba5920797c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:32.782197 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-e9456f9f-8c70-4b0a-aa1d-24ba5920797c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:32.783573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e9456f9f-8c70-4b0a-aa1d-24ba5920797c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:32.848375 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920aa4d8-975a-4f65-981e-6d566a182f82 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:32.857878 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d2f37-9358-4d17-a63a-8121f503f584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:32.859136 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d2f37-9358-4d17-a63a-8121f503f584 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:32.859643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d2f37-9358-4d17-a63a-8121f503f584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:32.860121 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d2f37-9358-4d17-a63a-8121f503f584 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:32.861537 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d2f37-9358-4d17-a63a-8121f503f584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:32.864607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d2f37-9358-4d17-a63a-8121f503f584 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:32.881154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 975/1918] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:32.887903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d2f37-9358-4d17-a63a-8121f503f584 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:32.890434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6d0d2f37-9358-4d17-a63a-8121f503f584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:32.911333 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d2f37-9358-4d17-a63a-8121f503f584 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:32.916420 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e150151-01dc-4286-8d57-226a9d08a5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:32.917131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e150151-01dc-4286-8d57-226a9d08a5d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:32.917419 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e150151-01dc-4286-8d57-226a9d08a5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:32.917698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e150151-01dc-4286-8d57-226a9d08a5d1 admin admin] PATH_INFO: `/v3/projects/7ec466531ece4564be190ad94ad765b5/users/1aa92e05305b45ae93a58016a235e860/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:32.922502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 944/1919] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:32.922880 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e150151-01dc-4286-8d57-226a9d08a5d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec466531ece4564be190ad94ad765b5, user_id=1aa92e05305b45ae93a58016a235e860, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:32.926246 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e150151-01dc-4286-8d57-226a9d08a5d1 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:32.936053 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5873b8-a50f-4942-b48b-cd1f5ee449fe None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:32.937933 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5873b8-a50f-4942-b48b-cd1f5ee449fe None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:32.938728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5873b8-a50f-4942-b48b-cd1f5ee449fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:32.944186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 976/1920] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:32 2022] PUT /identity/v3/projects/7ec466531ece4564be190ad94ad765b5/users/1aa92e05305b45ae93a58016a235e860/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:25:32.975058 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f533e-2a1d-4fbf-b1a2-c72439c2a8b3 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:32.975058 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f533e-2a1d-4fbf-b1a2-c72439c2a8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:32.975058 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f533e-2a1d-4fbf-b1a2-c72439c2a8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:32.991474 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-bd5873b8-a50f-4942-b48b-cd1f5ee449fe None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:32.993288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-bd5873b8-a50f-4942-b48b-cd1f5ee449fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:33.038983 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-595f533e-2a1d-4fbf-b1a2-c72439c2a8b3 None None] MFA Rules not processed for user `1aa92e05305b45ae93a58016a235e860`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:33.044224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-595f533e-2a1d-4fbf-b1a2-c72439c2a8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:33.161701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:33.182024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 945/1921] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:33.186472 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 977/1922] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:33.188120 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5873b8-a50f-4942-b48b-cd1f5ee449fe None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:33.197827 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f533e-2a1d-4fbf-b1a2-c72439c2a8b3 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:33.200846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-595f533e-2a1d-4fbf-b1a2-c72439c2a8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:33.207537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5873b8-a50f-4942-b48b-cd1f5ee449fe None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:33.226151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc095ca-dbb5-4d78-8dd4-e4b06847b115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:33.228068 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc095ca-dbb5-4d78-8dd4-e4b06847b115 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:33.229407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc095ca-dbb5-4d78-8dd4-e4b06847b115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:33.229819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc095ca-dbb5-4d78-8dd4-e4b06847b115 admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:33.235353 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc095ca-dbb5-4d78-8dd4-e4b06847b115 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:25:33.235780 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc095ca-dbb5-4d78-8dd4-e4b06847b115 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:33.240589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc095ca-dbb5-4d78-8dd4-e4b06847b115 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:33.282121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 946/1923] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:25:33.284007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc095ca-dbb5-4d78-8dd4-e4b06847b115 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:33.300369 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc095ca-dbb5-4d78-8dd4-e4b06847b115 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:33.313808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb9f01-cde3-470c-900c-604e225a647c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:33.314830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb9f01-cde3-470c-900c-604e225a647c admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:33.315327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb9f01-cde3-470c-900c-604e225a647c admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:33.315808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb9f01-cde3-470c-900c-604e225a647c admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:33.317451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb9f01-cde3-470c-900c-604e225a647c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:33.320351 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb9f01-cde3-470c-900c-604e225a647c admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:33.341620 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f533e-2a1d-4fbf-b1a2-c72439c2a8b3 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:33.360476 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7b842-4260-4038-b0eb-430d0c93f661 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:33.361475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7b842-4260-4038-b0eb-430d0c93f661 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:33.363468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7b842-4260-4038-b0eb-430d0c93f661 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:33.367876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7b842-4260-4038-b0eb-430d0c93f661 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:33.392330 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7b842-4260-4038-b0eb-430d0c93f661 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:33.395955 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7b842-4260-4038-b0eb-430d0c93f661 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:33.410182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:33.427114 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 978/1924] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:25:33.453244 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 947/1925] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:33 2022] POST /identity/v3/projects => generated 371 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:33.458786 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7b842-4260-4038-b0eb-430d0c93f661 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:33.478759 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7b842-4260-4038-b0eb-430d0c93f661 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:33.528318 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb9f01-cde3-470c-900c-604e225a647c admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:33.672463 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e07ca-8618-4ff8-88b3-a2153be5f0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:33.673786 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e07ca-8618-4ff8-88b3-a2153be5f0eb admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:33.674549 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e07ca-8618-4ff8-88b3-a2153be5f0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:33.675325 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e07ca-8618-4ff8-88b3-a2153be5f0eb admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:33.676286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e07ca-8618-4ff8-88b3-a2153be5f0eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:33.678321 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb9f01-cde3-470c-900c-604e225a647c admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:33.682796 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e07ca-8618-4ff8-88b3-a2153be5f0eb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:33.693191 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee9b8b-f4e9-48a7-b4d1-d598165a4581 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:33.694720 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee9b8b-f4e9-48a7-b4d1-d598165a4581 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:33.694924 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee9b8b-f4e9-48a7-b4d1-d598165a4581 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:33.695118 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee9b8b-f4e9-48a7-b4d1-d598165a4581 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:33.701026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c4ee9b8b-f4e9-48a7-b4d1-d598165a4581 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:33.734507 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 979/1926] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:33 2022] POST /identity/v3/users => generated 286 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:25:33.744428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e07ca-8618-4ff8-88b3-a2153be5f0eb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:33.774570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e07ca-8618-4ff8-88b3-a2153be5f0eb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:33.797654 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8a098-31c4-4906-8684-960ba7261600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:33.799665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8a098-31c4-4906-8684-960ba7261600 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:33.803772 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8a098-31c4-4906-8684-960ba7261600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:33.804785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8a098-31c4-4906-8684-960ba7261600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:33.807735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8a098-31c4-4906-8684-960ba7261600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:33.824025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8a098-31c4-4906-8684-960ba7261600 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:33.867673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee9b8b-f4e9-48a7-b4d1-d598165a4581 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:33.869295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee9b8b-f4e9-48a7-b4d1-d598165a4581 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:33.881470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 980/1927] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:33.885575 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8a098-31c4-4906-8684-960ba7261600 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:33.896016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 948/1928] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:25:33.939262 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8a098-31c4-4906-8684-960ba7261600 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:33.958144 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230166d-8374-4b66-87b7-c48794a7398e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:33.959809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230166d-8374-4b66-87b7-c48794a7398e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:33.967123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230166d-8374-4b66-87b7-c48794a7398e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:33.967123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230166d-8374-4b66-87b7-c48794a7398e admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:33.970798 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230166d-8374-4b66-87b7-c48794a7398e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:33.972734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230166d-8374-4b66-87b7-c48794a7398e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:34.029889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee9b8b-f4e9-48a7-b4d1-d598165a4581 service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:34.030103 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 981/1929] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:34.069893 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee9b8b-f4e9-48a7-b4d1-d598165a4581 service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:34.083825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cf845-97dc-48b0-9fb5-77b47fef30eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:34.084977 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cf845-97dc-48b0-9fb5-77b47fef30eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:34.086919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cf845-97dc-48b0-9fb5-77b47fef30eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:34.087791 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cf845-97dc-48b0-9fb5-77b47fef30eb admin admin] PATH_INFO: `/v3/projects/e5196561b7ae4adbb110ce06e81401f7/users/ba49197c9a3446b68bc92c48abf57b37/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:34.129261 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cf845-97dc-48b0-9fb5-77b47fef30eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5196561b7ae4adbb110ce06e81401f7, user_id=ba49197c9a3446b68bc92c48abf57b37, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:34.133180 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cf845-97dc-48b0-9fb5-77b47fef30eb admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:34.205080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 949/1930] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:34 2022] PUT /identity/v3/projects/e5196561b7ae4adbb110ce06e81401f7/users/ba49197c9a3446b68bc92c48abf57b37/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:34.212213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71944364-4132-4a4f-bf06-2eb3b72888e9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:34.212829 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71944364-4132-4a4f-bf06-2eb3b72888e9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:34.212956 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71944364-4132-4a4f-bf06-2eb3b72888e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:34.275446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-71944364-4132-4a4f-bf06-2eb3b72888e9 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:34.280728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-71944364-4132-4a4f-bf06-2eb3b72888e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:34.474300 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 982/1931] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:34.492350 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cf845-97dc-48b0-9fb5-77b47fef30eb admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:34.498636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-017cf845-97dc-48b0-9fb5-77b47fef30eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:34.592907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cf845-97dc-48b0-9fb5-77b47fef30eb admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:34.596824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7960d-f79c-4961-95b7-e9dbe1a6d422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:34.597516 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7960d-f79c-4961-95b7-e9dbe1a6d422 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:34.598035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7960d-f79c-4961-95b7-e9dbe1a6d422 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:34.598317 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7960d-f79c-4961-95b7-e9dbe1a6d422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:34.610092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:34.617259 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7960d-f79c-4961-95b7-e9dbe1a6d422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:34.619694 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7960d-f79c-4961-95b7-e9dbe1a6d422 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:34.638042 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 950/1932] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:36.667699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28593693-2ebf-4f3a-a6e5-ca231c4a9547 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:36.670050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28593693-2ebf-4f3a-a6e5-ca231c4a9547 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:36.670688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28593693-2ebf-4f3a-a6e5-ca231c4a9547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:36.747450 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-28593693-2ebf-4f3a-a6e5-ca231c4a9547 None None] MFA Rules not processed for user `ba49197c9a3446b68bc92c48abf57b37`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:36.750728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-28593693-2ebf-4f3a-a6e5-ca231c4a9547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:36.941657 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:36.965828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 983/1933] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:37.002299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7960d-f79c-4961-95b7-e9dbe1a6d422 service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:37.008270 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-48b7960d-f79c-4961-95b7-e9dbe1a6d422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:37.147583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7960d-f79c-4961-95b7-e9dbe1a6d422 service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:37.171407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1ccad-cf88-4f96-814e-9bdb9d8afe24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:37.172209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1ccad-cf88-4f96-814e-9bdb9d8afe24 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:37.172364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1ccad-cf88-4f96-814e-9bdb9d8afe24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:37.173362 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1ccad-cf88-4f96-814e-9bdb9d8afe24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:37.191119 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1ccad-cf88-4f96-814e-9bdb9d8afe24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:37.192590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1ccad-cf88-4f96-814e-9bdb9d8afe24 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:37.224990 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 951/1934] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:25:37.283806 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28593693-2ebf-4f3a-a6e5-ca231c4a9547 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:37.286439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-28593693-2ebf-4f3a-a6e5-ca231c4a9547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:37.328493 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28593693-2ebf-4f3a-a6e5-ca231c4a9547 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:37.342305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559437c-370f-4b4d-899a-35909c8ce9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:37.344125 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559437c-370f-4b4d-899a-35909c8ce9ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:37.344629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559437c-370f-4b4d-899a-35909c8ce9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:37.345764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559437c-370f-4b4d-899a-35909c8ce9ed admin admin] PATH_INFO: `/v3/users/18d73fd6ba0441289604e600edb47a4c` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:37.349926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559437c-370f-4b4d-899a-35909c8ce9ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=18d73fd6ba0441289604e600edb47a4c)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:37.352206 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559437c-370f-4b4d-899a-35909c8ce9ed admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:37.451052 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8559437c-370f-4b4d-899a-35909c8ce9ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18d73fd6ba0441289604e600edb47a4c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:37.468805 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8559437c-370f-4b4d-899a-35909c8ce9ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18d73fd6ba0441289604e600edb47a4c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:37.479726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8559437c-370f-4b4d-899a-35909c8ce9ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18d73fd6ba0441289604e600edb47a4c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:37.495477 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8559437c-370f-4b4d-899a-35909c8ce9ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18d73fd6ba0441289604e600edb47a4c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:37.522917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8559437c-370f-4b4d-899a-35909c8ce9ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18d73fd6ba0441289604e600edb47a4c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:37.537223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 984/1935] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:37 2022] DELETE /identity/v3/users/18d73fd6ba0441289604e600edb47a4c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:37.542188 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9885dc-a475-4c37-9c20-826e3878a430 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:37.542586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9885dc-a475-4c37-9c20-826e3878a430 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:37.543609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9885dc-a475-4c37-9c20-826e3878a430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:37.601987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-ae9885dc-a475-4c37-9c20-826e3878a430 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:37.603281 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ae9885dc-a475-4c37-9c20-826e3878a430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:37.729084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 952/1936] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:37.742477 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559437c-370f-4b4d-899a-35909c8ce9ed admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:37.745183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8559437c-370f-4b4d-899a-35909c8ce9ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:37.992451 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559437c-370f-4b4d-899a-35909c8ce9ed admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:38.004382 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9885dc-a475-4c37-9c20-826e3878a430 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:38.007395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ae9885dc-a475-4c37-9c20-826e3878a430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:38.008781 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f6958-b82f-4565-82e1-779e01271f7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:38.009526 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f6958-b82f-4565-82e1-779e01271f7c service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:38.009807 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f6958-b82f-4565-82e1-779e01271f7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:38.010071 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f6958-b82f-4565-82e1-779e01271f7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:38.021233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f6958-b82f-4565-82e1-779e01271f7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:38.023781 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f6958-b82f-4565-82e1-779e01271f7c service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:38.035576 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 985/1937] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:38.054115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9885dc-a475-4c37-9c20-826e3878a430 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:38.073497 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd459c7a-91c3-49ec-a2e9-16aa1bc6236e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:38.079137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd459c7a-91c3-49ec-a2e9-16aa1bc6236e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:38.079429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd459c7a-91c3-49ec-a2e9-16aa1bc6236e admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:38.079805 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd459c7a-91c3-49ec-a2e9-16aa1bc6236e admin admin] PATH_INFO: `/v3/users/d5ca1497d079419c93adf44549750181` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:38.081725 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd459c7a-91c3-49ec-a2e9-16aa1bc6236e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5ca1497d079419c93adf44549750181)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:38.087865 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd459c7a-91c3-49ec-a2e9-16aa1bc6236e admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:38.204092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-dd459c7a-91c3-49ec-a2e9-16aa1bc6236e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5ca1497d079419c93adf44549750181'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:38.204945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-dd459c7a-91c3-49ec-a2e9-16aa1bc6236e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5ca1497d079419c93adf44549750181'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:38.224367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-dd459c7a-91c3-49ec-a2e9-16aa1bc6236e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5ca1497d079419c93adf44549750181'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:38.274608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-dd459c7a-91c3-49ec-a2e9-16aa1bc6236e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5ca1497d079419c93adf44549750181'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:38.297673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-dd459c7a-91c3-49ec-a2e9-16aa1bc6236e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5ca1497d079419c93adf44549750181'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:38.324079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50328a22-b006-47b7-ade5-d9f08eb74757 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:38.324079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50328a22-b006-47b7-ade5-d9f08eb74757 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:38.325482 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50328a22-b006-47b7-ade5-d9f08eb74757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:38.328661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 953/1938] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:38 2022] DELETE /identity/v3/users/d5ca1497d079419c93adf44549750181 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:38.362202 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-50328a22-b006-47b7-ade5-d9f08eb74757 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:38.366600 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-50328a22-b006-47b7-ade5-d9f08eb74757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:38.525318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 986/1939] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:38.539290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd459c7a-91c3-49ec-a2e9-16aa1bc6236e admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:38.542121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-dd459c7a-91c3-49ec-a2e9-16aa1bc6236e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:38.562802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:38.649177 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd459c7a-91c3-49ec-a2e9-16aa1bc6236e admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:38.659348 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60440a-52bf-4f9c-9590-2be75072da76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:38.661813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60440a-52bf-4f9c-9590-2be75072da76 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:38.662577 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60440a-52bf-4f9c-9590-2be75072da76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:38.663561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60440a-52bf-4f9c-9590-2be75072da76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:38.693178 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60440a-52bf-4f9c-9590-2be75072da76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:38.698545 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60440a-52bf-4f9c-9590-2be75072da76 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:38.715945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 954/1940] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:25:38.952046 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50328a22-b006-47b7-ade5-d9f08eb74757 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:38.954066 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-50328a22-b006-47b7-ade5-d9f08eb74757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:38.971318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50328a22-b006-47b7-ade5-d9f08eb74757 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:38.977490 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188b774-bcc1-416c-be70-1d8d96eea6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:38.978467 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188b774-bcc1-416c-be70-1d8d96eea6a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:38.978847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188b774-bcc1-416c-be70-1d8d96eea6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:38.979214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188b774-bcc1-416c-be70-1d8d96eea6a4 admin admin] PATH_INFO: `/v3/projects/b28ed2601de241e2a3ce3d766032e962` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:38.980769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188b774-bcc1-416c-be70-1d8d96eea6a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b28ed2601de241e2a3ce3d766032e962)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:38.983195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188b774-bcc1-416c-be70-1d8d96eea6a4 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:39.007295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:39.087158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8188b774-bcc1-416c-be70-1d8d96eea6a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b28ed2601de241e2a3ce3d766032e962', 'request_id': 'req-8188b774-bcc1-416c-be70-1d8d96eea6a4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:39.096956 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8188b774-bcc1-416c-be70-1d8d96eea6a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b28ed2601de241e2a3ce3d766032e962', 'request_id': 'req-8188b774-bcc1-416c-be70-1d8d96eea6a4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:39.110344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8188b774-bcc1-416c-be70-1d8d96eea6a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b28ed2601de241e2a3ce3d766032e962', 'request_id': 'req-8188b774-bcc1-416c-be70-1d8d96eea6a4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:39.129949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8188b774-bcc1-416c-be70-1d8d96eea6a4 admin admin] The token cache is being invalidate because project b28ed2601de241e2a3ce3d766032e962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:39.129949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8188b774-bcc1-416c-be70-1d8d96eea6a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:39.129949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8188b774-bcc1-416c-be70-1d8d96eea6a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:39.130742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 987/1941] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:38 2022] DELETE /identity/v3/projects/b28ed2601de241e2a3ce3d766032e962 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:39.135059 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60440a-52bf-4f9c-9590-2be75072da76 service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:39.154127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60440a-52bf-4f9c-9590-2be75072da76 service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:39.197977 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156e0a4-58af-412e-854b-def5e5c88326 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:39.198167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156e0a4-58af-412e-854b-def5e5c88326 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:39.198427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156e0a4-58af-412e-854b-def5e5c88326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:39.220488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2156e0a4-58af-412e-854b-def5e5c88326 None None] MFA Rules not processed for user `ba49197c9a3446b68bc92c48abf57b37`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:39.221784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2156e0a4-58af-412e-854b-def5e5c88326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:39.237405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540c87d-cd89-435c-803d-a95562d6d707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:39.238793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540c87d-cd89-435c-803d-a95562d6d707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:39.239202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540c87d-cd89-435c-803d-a95562d6d707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:39.239630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540c87d-cd89-435c-803d-a95562d6d707 admin admin] PATH_INFO: `/v3/users/a428854028574a59a38aad240050be48` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:39.242630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540c87d-cd89-435c-803d-a95562d6d707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a428854028574a59a38aad240050be48)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:39.249271 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540c87d-cd89-435c-803d-a95562d6d707 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:39.325131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 988/1942] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:39.360090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6540c87d-cd89-435c-803d-a95562d6d707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a428854028574a59a38aad240050be48'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:39.362598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6540c87d-cd89-435c-803d-a95562d6d707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a428854028574a59a38aad240050be48'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:39.403799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6540c87d-cd89-435c-803d-a95562d6d707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a428854028574a59a38aad240050be48'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:39.404181 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156e0a4-58af-412e-854b-def5e5c88326 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:39.406115 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2156e0a4-58af-412e-854b-def5e5c88326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:39.417990 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6540c87d-cd89-435c-803d-a95562d6d707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a428854028574a59a38aad240050be48'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:39.438712 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6540c87d-cd89-435c-803d-a95562d6d707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a428854028574a59a38aad240050be48'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:39.461619 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 955/1943] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:39 2022] DELETE /identity/v3/users/a428854028574a59a38aad240050be48 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:39.461810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156e0a4-58af-412e-854b-def5e5c88326 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:39.663106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5474365-ca75-4f32-bc9e-057cda847a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:39.663402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5474365-ca75-4f32-bc9e-057cda847a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:39.663767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5474365-ca75-4f32-bc9e-057cda847a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:39.664135 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5474365-ca75-4f32-bc9e-057cda847a68 admin admin] PATH_INFO: `/v3/projects/b78c581171934f10abfd65dfc1cf90a0` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:39.665664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5474365-ca75-4f32-bc9e-057cda847a68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b78c581171934f10abfd65dfc1cf90a0)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:39.674053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5474365-ca75-4f32-bc9e-057cda847a68 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:39.827732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:39.848348 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f5474365-ca75-4f32-bc9e-057cda847a68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b78c581171934f10abfd65dfc1cf90a0', 'request_id': 'req-f5474365-ca75-4f32-bc9e-057cda847a68'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:39.865798 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f5474365-ca75-4f32-bc9e-057cda847a68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b78c581171934f10abfd65dfc1cf90a0', 'request_id': 'req-f5474365-ca75-4f32-bc9e-057cda847a68'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:39.882321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f5474365-ca75-4f32-bc9e-057cda847a68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b78c581171934f10abfd65dfc1cf90a0', 'request_id': 'req-f5474365-ca75-4f32-bc9e-057cda847a68'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:39.913728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f5474365-ca75-4f32-bc9e-057cda847a68 admin admin] The token cache is being invalidate because project b78c581171934f10abfd65dfc1cf90a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:39.914609 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f5474365-ca75-4f32-bc9e-057cda847a68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:39.918503 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f5474365-ca75-4f32-bc9e-057cda847a68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:39.924899 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 989/1944] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:39 2022] DELETE /identity/v3/projects/b78c581171934f10abfd65dfc1cf90a0 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:39.929845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540c87d-cd89-435c-803d-a95562d6d707 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:39.931459 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6540c87d-cd89-435c-803d-a95562d6d707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:39.950224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540c87d-cd89-435c-803d-a95562d6d707 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:40.013479 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:40.095519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c6c81-42fc-462a-b870-548cee498245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:40.096482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c6c81-42fc-462a-b870-548cee498245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:40.102580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c6c81-42fc-462a-b870-548cee498245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:40.103653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c6c81-42fc-462a-b870-548cee498245 admin admin] PATH_INFO: `/v3/users/fceab4f68b4c471290d95aa000f11474` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:40.107056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c6c81-42fc-462a-b870-548cee498245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fceab4f68b4c471290d95aa000f11474)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:40.111152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c6c81-42fc-462a-b870-548cee498245 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:40.214238 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-353c6c81-42fc-462a-b870-548cee498245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fceab4f68b4c471290d95aa000f11474'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:40.215734 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-353c6c81-42fc-462a-b870-548cee498245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fceab4f68b4c471290d95aa000f11474'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:40.232127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-353c6c81-42fc-462a-b870-548cee498245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fceab4f68b4c471290d95aa000f11474'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:40.252426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-353c6c81-42fc-462a-b870-548cee498245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fceab4f68b4c471290d95aa000f11474'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:40.266600 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-353c6c81-42fc-462a-b870-548cee498245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fceab4f68b4c471290d95aa000f11474'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:40.281138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 956/1945] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:39 2022] DELETE /identity/v3/users/fceab4f68b4c471290d95aa000f11474 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:40.645701 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5474365-ca75-4f32-bc9e-057cda847a68 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:40.648653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f5474365-ca75-4f32-bc9e-057cda847a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:40.690534 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5474365-ca75-4f32-bc9e-057cda847a68 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:40.838140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd08f0d-e1d0-40b2-8580-bbf0d992df31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:40.843082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd08f0d-e1d0-40b2-8580-bbf0d992df31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:40.843082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd08f0d-e1d0-40b2-8580-bbf0d992df31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:40.843082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd08f0d-e1d0-40b2-8580-bbf0d992df31 admin admin] PATH_INFO: `/v3/projects/89e3fa5077124564bf7478a134c808fa` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:40.845112 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd08f0d-e1d0-40b2-8580-bbf0d992df31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89e3fa5077124564bf7478a134c808fa)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:40.848991 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd08f0d-e1d0-40b2-8580-bbf0d992df31 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:40.939035 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-abd08f0d-e1d0-40b2-8580-bbf0d992df31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89e3fa5077124564bf7478a134c808fa', 'request_id': 'req-abd08f0d-e1d0-40b2-8580-bbf0d992df31'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:40.952211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-abd08f0d-e1d0-40b2-8580-bbf0d992df31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89e3fa5077124564bf7478a134c808fa', 'request_id': 'req-abd08f0d-e1d0-40b2-8580-bbf0d992df31'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:40.970506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-abd08f0d-e1d0-40b2-8580-bbf0d992df31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89e3fa5077124564bf7478a134c808fa', 'request_id': 'req-abd08f0d-e1d0-40b2-8580-bbf0d992df31'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:41.024770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-abd08f0d-e1d0-40b2-8580-bbf0d992df31 admin admin] The token cache is being invalidate because project 89e3fa5077124564bf7478a134c808fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:41.025631 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-abd08f0d-e1d0-40b2-8580-bbf0d992df31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:41.029002 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-abd08f0d-e1d0-40b2-8580-bbf0d992df31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:41.031602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 990/1946] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:40 2022] DELETE /identity/v3/projects/89e3fa5077124564bf7478a134c808fa => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:41.036657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a116af8-440e-41a8-96df-6bff299402b2 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:41.036958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a116af8-440e-41a8-96df-6bff299402b2 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:41.037250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a116af8-440e-41a8-96df-6bff299402b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:41.067947 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-0a116af8-440e-41a8-96df-6bff299402b2 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:41.069921 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0a116af8-440e-41a8-96df-6bff299402b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:41.186702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 957/1947] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:41.195400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57083000-5d9b-478d-94d5-4a25bbd1cf24 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:41.196827 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57083000-5d9b-478d-94d5-4a25bbd1cf24 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:41.197267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57083000-5d9b-478d-94d5-4a25bbd1cf24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:41.259619 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-57083000-5d9b-478d-94d5-4a25bbd1cf24 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:41.270192 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-57083000-5d9b-478d-94d5-4a25bbd1cf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:41.281185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a116af8-440e-41a8-96df-6bff299402b2 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:41.282889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0a116af8-440e-41a8-96df-6bff299402b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:41.307195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 991/1948] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:41.310590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a116af8-440e-41a8-96df-6bff299402b2 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:41.311800 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57083000-5d9b-478d-94d5-4a25bbd1cf24 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:41.333542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab364133-a4e2-410d-96ed-39ab969dee7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:41.334215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab364133-a4e2-410d-96ed-39ab969dee7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:41.334499 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab364133-a4e2-410d-96ed-39ab969dee7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:41.334775 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab364133-a4e2-410d-96ed-39ab969dee7a admin admin] PATH_INFO: `/v3/projects/dbfc2abbcadc438da6c154f2abaecae6` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:41.337168 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab364133-a4e2-410d-96ed-39ab969dee7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbfc2abbcadc438da6c154f2abaecae6)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:41.338588 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57083000-5d9b-478d-94d5-4a25bbd1cf24 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:41.339464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab364133-a4e2-410d-96ed-39ab969dee7a admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:41.348951 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42fc5b-3cb5-4c39-a7e2-a7b2e41c9400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:41.349866 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42fc5b-3cb5-4c39-a7e2-a7b2e41c9400 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:41.350325 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42fc5b-3cb5-4c39-a7e2-a7b2e41c9400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:41.350744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42fc5b-3cb5-4c39-a7e2-a7b2e41c9400 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:41.351769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42fc5b-3cb5-4c39-a7e2-a7b2e41c9400 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:25:41.352227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42fc5b-3cb5-4c39-a7e2-a7b2e41c9400 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:41.354289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42fc5b-3cb5-4c39-a7e2-a7b2e41c9400 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:41.381934 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 992/1949] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:25:41.390104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42fc5b-3cb5-4c39-a7e2-a7b2e41c9400 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:41.404693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42fc5b-3cb5-4c39-a7e2-a7b2e41c9400 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:41.410974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12516f33-dbf5-4fd8-bef6-6b3796e0db26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:41.411922 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12516f33-dbf5-4fd8-bef6-6b3796e0db26 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:41.412446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12516f33-dbf5-4fd8-bef6-6b3796e0db26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:41.412923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12516f33-dbf5-4fd8-bef6-6b3796e0db26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:41.414136 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12516f33-dbf5-4fd8-bef6-6b3796e0db26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:41.416525 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12516f33-dbf5-4fd8-bef6-6b3796e0db26 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:41.460592 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 993/1950] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:41 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:41.464843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12516f33-dbf5-4fd8-bef6-6b3796e0db26 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:41.470411 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:41.473313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-ab364133-a4e2-410d-96ed-39ab969dee7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbfc2abbcadc438da6c154f2abaecae6', 'request_id': 'req-ab364133-a4e2-410d-96ed-39ab969dee7a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:41.491473 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-ab364133-a4e2-410d-96ed-39ab969dee7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbfc2abbcadc438da6c154f2abaecae6', 'request_id': 'req-ab364133-a4e2-410d-96ed-39ab969dee7a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:41.505357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-ab364133-a4e2-410d-96ed-39ab969dee7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbfc2abbcadc438da6c154f2abaecae6', 'request_id': 'req-ab364133-a4e2-410d-96ed-39ab969dee7a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:41.541940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-ab364133-a4e2-410d-96ed-39ab969dee7a admin admin] The token cache is being invalidate because project dbfc2abbcadc438da6c154f2abaecae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:41.542570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-ab364133-a4e2-410d-96ed-39ab969dee7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:41.544853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-ab364133-a4e2-410d-96ed-39ab969dee7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:41.548949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:41.555202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 958/1951] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:41 2022] DELETE /identity/v3/projects/dbfc2abbcadc438da6c154f2abaecae6 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:41.567527 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab364133-a4e2-410d-96ed-39ab969dee7a admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:41.571001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ab364133-a4e2-410d-96ed-39ab969dee7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:41.711992 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12516f33-dbf5-4fd8-bef6-6b3796e0db26 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:41.738111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90eda8f-fc84-4ee0-950f-e7367403d848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:41.739657 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90eda8f-fc84-4ee0-950f-e7367403d848 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:41.740332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90eda8f-fc84-4ee0-950f-e7367403d848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:41.740798 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90eda8f-fc84-4ee0-950f-e7367403d848 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:41.742092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90eda8f-fc84-4ee0-950f-e7367403d848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:41.745939 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90eda8f-fc84-4ee0-950f-e7367403d848 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:41.767106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab364133-a4e2-410d-96ed-39ab969dee7a admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:41.786622 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5bfc7-055d-4914-8e23-894bfcb68c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:41.788417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5bfc7-055d-4914-8e23-894bfcb68c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:41.790529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5bfc7-055d-4914-8e23-894bfcb68c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:41.794847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5bfc7-055d-4914-8e23-894bfcb68c8e admin admin] PATH_INFO: `/v3/users/1aa92e05305b45ae93a58016a235e860` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:41.796892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5bfc7-055d-4914-8e23-894bfcb68c8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aa92e05305b45ae93a58016a235e860)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:41.803828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5bfc7-055d-4914-8e23-894bfcb68c8e admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:41.806734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 994/1952] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:41 2022] POST /identity/v3/users => generated 292 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:41.813005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90eda8f-fc84-4ee0-950f-e7367403d848 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:41.828679 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b90eda8f-fc84-4ee0-950f-e7367403d848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:41.846281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90eda8f-fc84-4ee0-950f-e7367403d848 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:41.853544 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33432446-8070-4d7e-8c66-fa83766cc75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:41.854216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33432446-8070-4d7e-8c66-fa83766cc75c admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:41.854527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33432446-8070-4d7e-8c66-fa83766cc75c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:41.854787 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33432446-8070-4d7e-8c66-fa83766cc75c admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:41.856053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33432446-8070-4d7e-8c66-fa83766cc75c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:41.858064 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33432446-8070-4d7e-8c66-fa83766cc75c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:41.875904 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 995/1953] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:41.881161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33432446-8070-4d7e-8c66-fa83766cc75c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:41.897553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33432446-8070-4d7e-8c66-fa83766cc75c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:41.902734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf34cd-9a0e-4959-8a7c-9a832b835e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:41.903506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf34cd-9a0e-4959-8a7c-9a832b835e6d admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:41.903995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf34cd-9a0e-4959-8a7c-9a832b835e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:41.904267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf34cd-9a0e-4959-8a7c-9a832b835e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:41.904920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf34cd-9a0e-4959-8a7c-9a832b835e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:41.906687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf34cd-9a0e-4959-8a7c-9a832b835e6d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:41.912174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5bb5bfc7-055d-4914-8e23-894bfcb68c8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aa92e05305b45ae93a58016a235e860'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:41.912884 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5bb5bfc7-055d-4914-8e23-894bfcb68c8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aa92e05305b45ae93a58016a235e860'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:41.922841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5bb5bfc7-055d-4914-8e23-894bfcb68c8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aa92e05305b45ae93a58016a235e860'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:41.939358 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5bb5bfc7-055d-4914-8e23-894bfcb68c8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aa92e05305b45ae93a58016a235e860'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:41.939805 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 996/1954] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:25:41.940770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf34cd-9a0e-4959-8a7c-9a832b835e6d admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:41.950378 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5bb5bfc7-055d-4914-8e23-894bfcb68c8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aa92e05305b45ae93a58016a235e860'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:41.960976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 959/1955] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:41 2022] DELETE /identity/v3/users/1aa92e05305b45ae93a58016a235e860 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:41.967680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf34cd-9a0e-4959-8a7c-9a832b835e6d admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:42.085944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786335cf-5f10-40d5-a026-476ae04e84a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:42.087289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786335cf-5f10-40d5-a026-476ae04e84a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:42.087656 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786335cf-5f10-40d5-a026-476ae04e84a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:42.089171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786335cf-5f10-40d5-a026-476ae04e84a2 admin admin] PATH_INFO: `/v3/projects/22c10af3726e4d9189d5c8ba3454c86c/users/fbd2dc2367a14fe78f7344841b901746/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:42.112595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786335cf-5f10-40d5-a026-476ae04e84a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22c10af3726e4d9189d5c8ba3454c86c, user_id=fbd2dc2367a14fe78f7344841b901746, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:42.116283 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786335cf-5f10-40d5-a026-476ae04e84a2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:42.136518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 997/1956] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:41 2022] PUT /identity/v3/projects/22c10af3726e4d9189d5c8ba3454c86c/users/fbd2dc2367a14fe78f7344841b901746/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:42.149295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803f6d4-8b3f-4694-b440-45bd9b4195af None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:42.149760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803f6d4-8b3f-4694-b440-45bd9b4195af None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:42.149914 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803f6d4-8b3f-4694-b440-45bd9b4195af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:42.223517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-0803f6d4-8b3f-4694-b440-45bd9b4195af None None] MFA Rules not processed for user `fbd2dc2367a14fe78f7344841b901746`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:42.225019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0803f6d4-8b3f-4694-b440-45bd9b4195af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:42.350308 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:42.390004 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 960/1957] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:42.390742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786335cf-5f10-40d5-a026-476ae04e84a2 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:42.394759 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-786335cf-5f10-40d5-a026-476ae04e84a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:42.518779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786335cf-5f10-40d5-a026-476ae04e84a2 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:42.526568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa696b5-4586-4050-9e46-62503afebc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:42.528822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa696b5-4586-4050-9e46-62503afebc24 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:42.529314 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa696b5-4586-4050-9e46-62503afebc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:42.529671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa696b5-4586-4050-9e46-62503afebc24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:42.530439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa696b5-4586-4050-9e46-62503afebc24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:42.532855 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa696b5-4586-4050-9e46-62503afebc24 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:42.590336 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:42.653735 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803f6d4-8b3f-4694-b440-45bd9b4195af None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:42.663677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 998/1958] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:42 2022] POST /identity/v3/projects => generated 385 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:25:42.670958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803f6d4-8b3f-4694-b440-45bd9b4195af None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:42.754577 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa696b5-4586-4050-9e46-62503afebc24 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:42.756808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ffa696b5-4586-4050-9e46-62503afebc24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:42.831931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420ce0e-733a-4ff9-a694-76583640c19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:42.832552 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420ce0e-733a-4ff9-a694-76583640c19a admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:42.832835 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420ce0e-733a-4ff9-a694-76583640c19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:42.833136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420ce0e-733a-4ff9-a694-76583640c19a admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:42.833789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420ce0e-733a-4ff9-a694-76583640c19a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:42.835792 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420ce0e-733a-4ff9-a694-76583640c19a admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:42.866488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 961/1959] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:42 2022] POST /identity/v3/users => generated 299 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:42.874343 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420ce0e-733a-4ff9-a694-76583640c19a admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:42.893810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420ce0e-733a-4ff9-a694-76583640c19a admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:42.902332 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370ae60-c6e6-4d31-97a4-983cf195d5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:42.903988 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370ae60-c6e6-4d31-97a4-983cf195d5e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:42.904269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370ae60-c6e6-4d31-97a4-983cf195d5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:42.904546 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370ae60-c6e6-4d31-97a4-983cf195d5e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:42.905562 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370ae60-c6e6-4d31-97a4-983cf195d5e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:42.907314 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370ae60-c6e6-4d31-97a4-983cf195d5e2 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:42.907633 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa696b5-4586-4050-9e46-62503afebc24 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:42.920371 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824d0bd-abd1-43ec-bb8f-8e3c77cff128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:42.921519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824d0bd-abd1-43ec-bb8f-8e3c77cff128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:42.921803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824d0bd-abd1-43ec-bb8f-8e3c77cff128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:42.921927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 962/1960] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:42.922080 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824d0bd-abd1-43ec-bb8f-8e3c77cff128 admin admin] PATH_INFO: `/v3/projects/7ec466531ece4564be190ad94ad765b5` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:42.925528 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824d0bd-abd1-43ec-bb8f-8e3c77cff128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec466531ece4564be190ad94ad765b5)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:42.927961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824d0bd-abd1-43ec-bb8f-8e3c77cff128 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:42.937587 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370ae60-c6e6-4d31-97a4-983cf195d5e2 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:42.977437 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370ae60-c6e6-4d31-97a4-983cf195d5e2 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:42.983348 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329fdb3-052a-45a6-9320-1b9ae4d7a383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:42.984369 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329fdb3-052a-45a6-9320-1b9ae4d7a383 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:42.984843 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329fdb3-052a-45a6-9320-1b9ae4d7a383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:42.985577 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329fdb3-052a-45a6-9320-1b9ae4d7a383 admin admin] PATH_INFO: `/v3/projects/090b210e207f4446b1e06cdccc75b1d1/users/379c68a4eaf94663a713fb4814d818d4/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:43.011254 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8329fdb3-052a-45a6-9320-1b9ae4d7a383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=090b210e207f4446b1e06cdccc75b1d1, user_id=379c68a4eaf94663a713fb4814d818d4, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:43.015438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8329fdb3-052a-45a6-9320-1b9ae4d7a383 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:43.058200 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 963/1961] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:42 2022] PUT /identity/v3/projects/090b210e207f4446b1e06cdccc75b1d1/users/379c68a4eaf94663a713fb4814d818d4/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:43.060599 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329fdb3-052a-45a6-9320-1b9ae4d7a383 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:43.082598 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8824d0bd-abd1-43ec-bb8f-8e3c77cff128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec466531ece4564be190ad94ad765b5', 'request_id': 'req-8824d0bd-abd1-43ec-bb8f-8e3c77cff128'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:43.104188 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8824d0bd-abd1-43ec-bb8f-8e3c77cff128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec466531ece4564be190ad94ad765b5', 'request_id': 'req-8824d0bd-abd1-43ec-bb8f-8e3c77cff128'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:43.120621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8824d0bd-abd1-43ec-bb8f-8e3c77cff128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec466531ece4564be190ad94ad765b5', 'request_id': 'req-8824d0bd-abd1-43ec-bb8f-8e3c77cff128'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:43.176754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8824d0bd-abd1-43ec-bb8f-8e3c77cff128 admin admin] The token cache is being invalidate because project 7ec466531ece4564be190ad94ad765b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:43.178555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8824d0bd-abd1-43ec-bb8f-8e3c77cff128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:43.178743 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329fdb3-052a-45a6-9320-1b9ae4d7a383 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:43.184264 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8824d0bd-abd1-43ec-bb8f-8e3c77cff128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:43.190827 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 999/1962] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:42 2022] DELETE /identity/v3/projects/7ec466531ece4564be190ad94ad765b5 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:43.192953 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdd85a-118b-43f3-8577-6ca4cc8b5434 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:43.193370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdd85a-118b-43f3-8577-6ca4cc8b5434 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:43.193675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdd85a-118b-43f3-8577-6ca4cc8b5434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:43.242308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0efdd85a-118b-43f3-8577-6ca4cc8b5434 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:43.245476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:43.246925 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0efdd85a-118b-43f3-8577-6ca4cc8b5434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:43.267019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e833d-1694-4e2e-9214-86b4bec458f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:43.268227 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e833d-1694-4e2e-9214-86b4bec458f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:43.268666 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e833d-1694-4e2e-9214-86b4bec458f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:43.269196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e833d-1694-4e2e-9214-86b4bec458f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:43.270250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e833d-1694-4e2e-9214-86b4bec458f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:43.273298 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e833d-1694-4e2e-9214-86b4bec458f3 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:43.295514 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 964/1963] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:43.300784 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e833d-1694-4e2e-9214-86b4bec458f3 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:43.303571 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-840e833d-1694-4e2e-9214-86b4bec458f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:43.346502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e833d-1694-4e2e-9214-86b4bec458f3 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:43.357910 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1bff6-890d-4f9b-9bce-7d4f0da2376c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:43.360168 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1bff6-890d-4f9b-9bce-7d4f0da2376c admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:43.360705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1bff6-890d-4f9b-9bce-7d4f0da2376c admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:43.360992 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1bff6-890d-4f9b-9bce-7d4f0da2376c admin admin] PATH_INFO: `/v3/projects/090b210e207f4446b1e06cdccc75b1d1/users/379c68a4eaf94663a713fb4814d818d4/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:43.363697 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1bff6-890d-4f9b-9bce-7d4f0da2376c admin admin] RBAC: Authorizing `identity:create_grant(project_id=090b210e207f4446b1e06cdccc75b1d1, user_id=379c68a4eaf94663a713fb4814d818d4, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:43.368470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1bff6-890d-4f9b-9bce-7d4f0da2376c admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:43.397831 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 965/1964] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:43 2022] PUT /identity/v3/projects/090b210e207f4446b1e06cdccc75b1d1/users/379c68a4eaf94663a713fb4814d818d4/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:43.415375 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c7c40-48f6-427f-b073-daa320b647ff None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:43.415784 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c7c40-48f6-427f-b073-daa320b647ff None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:43.415965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c7c40-48f6-427f-b073-daa320b647ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:43.443906 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1000/1965] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:43.454249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ccdef-e8a3-4f08-8966-0dbc9cee650c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:43.454595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ccdef-e8a3-4f08-8966-0dbc9cee650c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:43.454887 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ccdef-e8a3-4f08-8966-0dbc9cee650c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:43.498909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-3b0c7c40-48f6-427f-b073-daa320b647ff None None] MFA Rules not processed for user `379c68a4eaf94663a713fb4814d818d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:43.502654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3b0c7c40-48f6-427f-b073-daa320b647ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:43.530852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d28ccdef-e8a3-4f08-8966-0dbc9cee650c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:43.532690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d28ccdef-e8a3-4f08-8966-0dbc9cee650c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:43.601806 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:43.703510 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 966/1966] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:43.708065 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1001/1967] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:43.712582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c7c40-48f6-427f-b073-daa320b647ff None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:43.725787 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ccdef-e8a3-4f08-8966-0dbc9cee650c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:43.728679 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d28ccdef-e8a3-4f08-8966-0dbc9cee650c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:43.729143 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c7c40-48f6-427f-b073-daa320b647ff None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:43.751896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7f925-079f-405b-85ae-d62a7c8e6787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:43.766825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7f925-079f-405b-85ae-d62a7c8e6787 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:43.767662 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7f925-079f-405b-85ae-d62a7c8e6787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:43.768252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7f925-079f-405b-85ae-d62a7c8e6787 admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:43.780526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7f925-079f-405b-85ae-d62a7c8e6787 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:25:43.789560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7f925-079f-405b-85ae-d62a7c8e6787 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:43.793926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7f925-079f-405b-85ae-d62a7c8e6787 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:43.837528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 967/1968] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:25:43.840788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7f925-079f-405b-85ae-d62a7c8e6787 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:43.870671 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7f925-079f-405b-85ae-d62a7c8e6787 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:43.876757 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6fd18-1047-4d18-b033-45546d842500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:43.877433 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6fd18-1047-4d18-b033-45546d842500 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:43.877716 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6fd18-1047-4d18-b033-45546d842500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:43.878009 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6fd18-1047-4d18-b033-45546d842500 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:43.878893 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6fd18-1047-4d18-b033-45546d842500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:43.881148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6fd18-1047-4d18-b033-45546d842500 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:43.952487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ccdef-e8a3-4f08-8966-0dbc9cee650c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:43.955496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 968/1969] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:43 2022] POST /identity/v3/projects => generated 367 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:43.961640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6fd18-1047-4d18-b033-45546d842500 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:44.148633 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04597a9b-a1b7-475f-8362-f6b8d0720122 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:44.149819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04597a9b-a1b7-475f-8362-f6b8d0720122 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:44.150485 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04597a9b-a1b7-475f-8362-f6b8d0720122 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:44.150993 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04597a9b-a1b7-475f-8362-f6b8d0720122 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:44.155832 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-04597a9b-a1b7-475f-8362-f6b8d0720122 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:44.163757 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6fd18-1047-4d18-b033-45546d842500 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:44.178180 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcdbfc-f43a-4186-80aa-429f62000230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:44.178977 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcdbfc-f43a-4186-80aa-429f62000230 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:44.179366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcdbfc-f43a-4186-80aa-429f62000230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:44.180686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcdbfc-f43a-4186-80aa-429f62000230 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:44.181566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dcdbfc-f43a-4186-80aa-429f62000230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:44.186429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dcdbfc-f43a-4186-80aa-429f62000230 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:44.243312 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 969/1970] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:43 2022] POST /identity/v3/users => generated 284 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:44.248799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcdbfc-f43a-4186-80aa-429f62000230 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:44.285563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcdbfc-f43a-4186-80aa-429f62000230 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:44.298492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d961b-eba2-419d-8b91-a28a320c614d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:44.301445 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d961b-eba2-419d-8b91-a28a320c614d admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:44.301788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d961b-eba2-419d-8b91-a28a320c614d admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:44.302820 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d961b-eba2-419d-8b91-a28a320c614d admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:44.303526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d961b-eba2-419d-8b91-a28a320c614d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:44.306554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d961b-eba2-419d-8b91-a28a320c614d admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:44.316007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:44.325171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04597a9b-a1b7-475f-8362-f6b8d0720122 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:44.327388 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04597a9b-a1b7-475f-8362-f6b8d0720122 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:44.355596 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 970/1971] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:44.359639 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1002/1972] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:44.360949 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d961b-eba2-419d-8b91-a28a320c614d admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:44.384644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d961b-eba2-419d-8b91-a28a320c614d admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:44.402243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79294004-0415-46dd-b699-9f192061e823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:44.403887 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79294004-0415-46dd-b699-9f192061e823 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:44.404181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79294004-0415-46dd-b699-9f192061e823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:44.404424 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79294004-0415-46dd-b699-9f192061e823 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:44.405089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79294004-0415-46dd-b699-9f192061e823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:44.406787 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79294004-0415-46dd-b699-9f192061e823 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:44.443570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 971/1973] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:25:44.444973 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04597a9b-a1b7-475f-8362-f6b8d0720122 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:44.478077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04597a9b-a1b7-475f-8362-f6b8d0720122 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:44.487005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daee30a-3790-49f4-ae4c-d047a9401636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:44.487810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daee30a-3790-49f4-ae4c-d047a9401636 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:44.488102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daee30a-3790-49f4-ae4c-d047a9401636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:44.488383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daee30a-3790-49f4-ae4c-d047a9401636 admin admin] PATH_INFO: `/v3/projects/90894beb482a44dd9abb213330815d5f/users/a12ab2e2d2384282bdbbe3c2518e4962/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:44.539045 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daee30a-3790-49f4-ae4c-d047a9401636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90894beb482a44dd9abb213330815d5f, user_id=a12ab2e2d2384282bdbbe3c2518e4962, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:44.549838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daee30a-3790-49f4-ae4c-d047a9401636 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:44.587929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1003/1974] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:44 2022] PUT /identity/v3/projects/90894beb482a44dd9abb213330815d5f/users/a12ab2e2d2384282bdbbe3c2518e4962/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:44.603633 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86485f1-653b-4955-9086-6e31d47775c9 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:44.603863 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86485f1-653b-4955-9086-6e31d47775c9 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:44.604285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86485f1-653b-4955-9086-6e31d47775c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:44.664712 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-d86485f1-653b-4955-9086-6e31d47775c9 None None] MFA Rules not processed for user `a12ab2e2d2384282bdbbe3c2518e4962`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:44.668257 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d86485f1-653b-4955-9086-6e31d47775c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:44.896042 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 972/1975] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:44.898317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daee30a-3790-49f4-ae4c-d047a9401636 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:44.978574 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:45.143888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daee30a-3790-49f4-ae4c-d047a9401636 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:45.167810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2e332-8ebb-4a30-9bc7-b62e0951d23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:45.168687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2e332-8ebb-4a30-9bc7-b62e0951d23a admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:45.169373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2e332-8ebb-4a30-9bc7-b62e0951d23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:45.170147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2e332-8ebb-4a30-9bc7-b62e0951d23a admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:45.175206 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2e332-8ebb-4a30-9bc7-b62e0951d23a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:45.180930 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2e332-8ebb-4a30-9bc7-b62e0951d23a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:45.252818 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1004/1976] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:44 2022] POST /identity/v3/projects => generated 359 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:45.256699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86485f1-653b-4955-9086-6e31d47775c9 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:45.274135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86485f1-653b-4955-9086-6e31d47775c9 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:45.392943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf50275-c45f-4775-b7f6-50b64331f248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:45.393812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf50275-c45f-4775-b7f6-50b64331f248 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:45.394224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf50275-c45f-4775-b7f6-50b64331f248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:45.394665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf50275-c45f-4775-b7f6-50b64331f248 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:45.395536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf50275-c45f-4775-b7f6-50b64331f248 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:45.398256 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf50275-c45f-4775-b7f6-50b64331f248 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:45.430238 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 973/1977] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:45 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:45.435342 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2e332-8ebb-4a30-9bc7-b62e0951d23a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:45.457852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2e332-8ebb-4a30-9bc7-b62e0951d23a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:45.468272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51943a1-b6aa-40d5-a52a-ab8bf8f4f442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:45.469191 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51943a1-b6aa-40d5-a52a-ab8bf8f4f442 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:45.469555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51943a1-b6aa-40d5-a52a-ab8bf8f4f442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:45.469954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51943a1-b6aa-40d5-a52a-ab8bf8f4f442 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:45.471954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51943a1-b6aa-40d5-a52a-ab8bf8f4f442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:45.475962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51943a1-b6aa-40d5-a52a-ab8bf8f4f442 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:45.507662 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1005/1978] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:45.513312 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf50275-c45f-4775-b7f6-50b64331f248 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:45.525756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf50275-c45f-4775-b7f6-50b64331f248 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:45.532815 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a5ac4-8b69-45a3-afd1-cc3a5d0a3177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:45.534145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a5ac4-8b69-45a3-afd1-cc3a5d0a3177 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:45.534663 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a5ac4-8b69-45a3-afd1-cc3a5d0a3177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:45.535186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a5ac4-8b69-45a3-afd1-cc3a5d0a3177 admin admin] PATH_INFO: `/v3/projects/8cfc5abf8dbc4b2483cec34cd0fe9fc0/users/a19b4c92954447fba89e7ae494c0856d/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:45.555391 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a5ac4-8b69-45a3-afd1-cc3a5d0a3177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cfc5abf8dbc4b2483cec34cd0fe9fc0, user_id=a19b4c92954447fba89e7ae494c0856d, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:45.560949 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a5ac4-8b69-45a3-afd1-cc3a5d0a3177 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:45.601808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 974/1979] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:45 2022] PUT /identity/v3/projects/8cfc5abf8dbc4b2483cec34cd0fe9fc0/users/a19b4c92954447fba89e7ae494c0856d/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:45.607475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51943a1-b6aa-40d5-a52a-ab8bf8f4f442 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:45.649709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51943a1-b6aa-40d5-a52a-ab8bf8f4f442 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:45.836365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8e984-800b-4186-853d-84c2a64bdbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:45.836734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8e984-800b-4186-853d-84c2a64bdbb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:45.837019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8e984-800b-4186-853d-84c2a64bdbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:45.837352 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8e984-800b-4186-853d-84c2a64bdbb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:45.838014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8e984-800b-4186-853d-84c2a64bdbb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:45.839714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8e984-800b-4186-853d-84c2a64bdbb1 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:45.854848 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1006/1980] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:45.859345 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a5ac4-8b69-45a3-afd1-cc3a5d0a3177 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:45.883153 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a5ac4-8b69-45a3-afd1-cc3a5d0a3177 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:45.897843 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337b11d-f62a-42e4-91bb-625065a3f6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:45.905586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337b11d-f62a-42e4-91bb-625065a3f6d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:45.906807 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337b11d-f62a-42e4-91bb-625065a3f6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:45.907509 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337b11d-f62a-42e4-91bb-625065a3f6d6 admin admin] PATH_INFO: `/v3/projects/8cfc5abf8dbc4b2483cec34cd0fe9fc0/users/a19b4c92954447fba89e7ae494c0856d/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:45.914810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337b11d-f62a-42e4-91bb-625065a3f6d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cfc5abf8dbc4b2483cec34cd0fe9fc0, user_id=a19b4c92954447fba89e7ae494c0856d, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:45.923653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337b11d-f62a-42e4-91bb-625065a3f6d6 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:45.952474 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 975/1981] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:45 2022] PUT /identity/v3/projects/8cfc5abf8dbc4b2483cec34cd0fe9fc0/users/a19b4c92954447fba89e7ae494c0856d/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:45.953623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecf720-431a-41da-a32d-e36895fdd3b2 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:45.955382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecf720-431a-41da-a32d-e36895fdd3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:45.955815 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecf720-431a-41da-a32d-e36895fdd3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:46.050260 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-35ecf720-431a-41da-a32d-e36895fdd3b2 None None] MFA Rules not processed for user `a19b4c92954447fba89e7ae494c0856d`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:46.054061 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-35ecf720-431a-41da-a32d-e36895fdd3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:46.272305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1007/1982] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:45 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:46.314939 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337b11d-f62a-42e4-91bb-625065a3f6d6 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:46.472911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:46.549868 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337b11d-f62a-42e4-91bb-625065a3f6d6 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:46.556093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21830b38-c497-4977-a470-bfd32a38db23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:46.565584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21830b38-c497-4977-a470-bfd32a38db23 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:46.565584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21830b38-c497-4977-a470-bfd32a38db23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:46.565584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21830b38-c497-4977-a470-bfd32a38db23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:46.613342 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21830b38-c497-4977-a470-bfd32a38db23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:46.617351 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21830b38-c497-4977-a470-bfd32a38db23 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:46.643649 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 976/1983] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:46 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:46.877387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecf720-431a-41da-a32d-e36895fdd3b2 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:46.890483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecf720-431a-41da-a32d-e36895fdd3b2 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:46.900938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4afdcf-2dc6-46e2-b95e-83dc68c455ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:46.902258 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4afdcf-2dc6-46e2-b95e-83dc68c455ba service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:46.902772 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4afdcf-2dc6-46e2-b95e-83dc68c455ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:46.902981 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4afdcf-2dc6-46e2-b95e-83dc68c455ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:46.917124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:47.076002 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4afdcf-2dc6-46e2-b95e-83dc68c455ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:47.085224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4afdcf-2dc6-46e2-b95e-83dc68c455ba service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:47.112018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1008/1984] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:25:50.325282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21830b38-c497-4977-a470-bfd32a38db23 service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:50.325282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-21830b38-c497-4977-a470-bfd32a38db23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:50.347326 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21830b38-c497-4977-a470-bfd32a38db23 service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:50.354208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6b3e4-0735-4580-a6df-03b0fb12eda4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:50.355476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6b3e4-0735-4580-a6df-03b0fb12eda4 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:50.356190 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6b3e4-0735-4580-a6df-03b0fb12eda4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:50.356754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6b3e4-0735-4580-a6df-03b0fb12eda4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:50.447789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6b3e4-0735-4580-a6df-03b0fb12eda4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:50.451529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6b3e4-0735-4580-a6df-03b0fb12eda4 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:50.464064 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 977/1985] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:53.268179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4afdcf-2dc6-46e2-b95e-83dc68c455ba service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:53.381101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4afdcf-2dc6-46e2-b95e-83dc68c455ba service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:53.387459 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646e071-8321-4d28-81dc-e59f14b61c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:53.388471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646e071-8321-4d28-81dc-e59f14b61c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:53.388879 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646e071-8321-4d28-81dc-e59f14b61c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:53.389337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646e071-8321-4d28-81dc-e59f14b61c4b admin admin] PATH_INFO: `/v3/users/a12ab2e2d2384282bdbbe3c2518e4962` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:53.390995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646e071-8321-4d28-81dc-e59f14b61c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a12ab2e2d2384282bdbbe3c2518e4962)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:53.393136 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646e071-8321-4d28-81dc-e59f14b61c4b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:53.466305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5646e071-8321-4d28-81dc-e59f14b61c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a12ab2e2d2384282bdbbe3c2518e4962'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:53.484768 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5646e071-8321-4d28-81dc-e59f14b61c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a12ab2e2d2384282bdbbe3c2518e4962'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:53.495474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5646e071-8321-4d28-81dc-e59f14b61c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a12ab2e2d2384282bdbbe3c2518e4962'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:53.496545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5646e071-8321-4d28-81dc-e59f14b61c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a12ab2e2d2384282bdbbe3c2518e4962'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:53.511077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5646e071-8321-4d28-81dc-e59f14b61c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a12ab2e2d2384282bdbbe3c2518e4962'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:53.524498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1009/1986] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:53 2022] DELETE /identity/v3/users/a12ab2e2d2384282bdbbe3c2518e4962 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:53.531860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62b28a-4bc8-4553-9193-914cdeccb4f0 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:53.532460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62b28a-4bc8-4553-9193-914cdeccb4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:53.532460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62b28a-4bc8-4553-9193-914cdeccb4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:53.555076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-4b62b28a-4bc8-4553-9193-914cdeccb4f0 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:53.556306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4b62b28a-4bc8-4553-9193-914cdeccb4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:53.657430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 978/1987] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:53.679306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646e071-8321-4d28-81dc-e59f14b61c4b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:53.680960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5646e071-8321-4d28-81dc-e59f14b61c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:53.724879 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:53.779294 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646e071-8321-4d28-81dc-e59f14b61c4b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:53.791555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526424a-280a-49f3-90cf-d3caf5718656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:53.794227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526424a-280a-49f3-90cf-d3caf5718656 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:53.794227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526424a-280a-49f3-90cf-d3caf5718656 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:53.794227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526424a-280a-49f3-90cf-d3caf5718656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:53.812885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526424a-280a-49f3-90cf-d3caf5718656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:53.814742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526424a-280a-49f3-90cf-d3caf5718656 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:53.830399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1010/1988] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:54.249829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62b28a-4bc8-4553-9193-914cdeccb4f0 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:54.250951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4b62b28a-4bc8-4553-9193-914cdeccb4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:54.273307 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62b28a-4bc8-4553-9193-914cdeccb4f0 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:54.283234 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03d99c-a27c-4110-a919-a4764e69c0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:54.284938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03d99c-a27c-4110-a919-a4764e69c0da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:54.285852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03d99c-a27c-4110-a919-a4764e69c0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:54.286753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03d99c-a27c-4110-a919-a4764e69c0da admin admin] PATH_INFO: `/v3/projects/90894beb482a44dd9abb213330815d5f` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:54.288695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03d99c-a27c-4110-a919-a4764e69c0da admin admin] RBAC: Authorizing `identity:delete_project(project_id=90894beb482a44dd9abb213330815d5f)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:54.291433 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03d99c-a27c-4110-a919-a4764e69c0da admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:54.353198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f03d99c-a27c-4110-a919-a4764e69c0da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90894beb482a44dd9abb213330815d5f', 'request_id': 'req-4f03d99c-a27c-4110-a919-a4764e69c0da'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:54.360692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f03d99c-a27c-4110-a919-a4764e69c0da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90894beb482a44dd9abb213330815d5f', 'request_id': 'req-4f03d99c-a27c-4110-a919-a4764e69c0da'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:54.366702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:54.368356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f03d99c-a27c-4110-a919-a4764e69c0da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90894beb482a44dd9abb213330815d5f', 'request_id': 'req-4f03d99c-a27c-4110-a919-a4764e69c0da'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:54.387629 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f03d99c-a27c-4110-a919-a4764e69c0da admin admin] The token cache is being invalidate because project 90894beb482a44dd9abb213330815d5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:54.387885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f03d99c-a27c-4110-a919-a4764e69c0da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:54.388801 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f03d99c-a27c-4110-a919-a4764e69c0da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:54.390863 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 979/1989] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:54 2022] DELETE /identity/v3/projects/90894beb482a44dd9abb213330815d5f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:54.394973 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526424a-280a-49f3-90cf-d3caf5718656 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:54.410470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526424a-280a-49f3-90cf-d3caf5718656 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:54.501199 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ff2e1-fec7-4145-90b3-3c2f830025d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:54.501844 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ff2e1-fec7-4145-90b3-3c2f830025d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:54.502029 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ff2e1-fec7-4145-90b3-3c2f830025d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:54.502337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ff2e1-fec7-4145-90b3-3c2f830025d9 admin admin] PATH_INFO: `/v3/users/a19b4c92954447fba89e7ae494c0856d` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:54.505926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ff2e1-fec7-4145-90b3-3c2f830025d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a19b4c92954447fba89e7ae494c0856d)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:54.507833 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ff2e1-fec7-4145-90b3-3c2f830025d9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:54.584750 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-069ff2e1-fec7-4145-90b3-3c2f830025d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a19b4c92954447fba89e7ae494c0856d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:54.604026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-069ff2e1-fec7-4145-90b3-3c2f830025d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a19b4c92954447fba89e7ae494c0856d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:54.613883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-069ff2e1-fec7-4145-90b3-3c2f830025d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a19b4c92954447fba89e7ae494c0856d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:54.614945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-069ff2e1-fec7-4145-90b3-3c2f830025d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a19b4c92954447fba89e7ae494c0856d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:54.629297 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-069ff2e1-fec7-4145-90b3-3c2f830025d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a19b4c92954447fba89e7ae494c0856d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:54.639370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1011/1990] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:54 2022] DELETE /identity/v3/users/a19b4c92954447fba89e7ae494c0856d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:55.034783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03d99c-a27c-4110-a919-a4764e69c0da admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:55.038587 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4f03d99c-a27c-4110-a919-a4764e69c0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:55.069093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03d99c-a27c-4110-a919-a4764e69c0da admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:55.140952 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9814bba-f3fc-4224-9863-ede8f5a186f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:55.141583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9814bba-f3fc-4224-9863-ede8f5a186f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:55.141864 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9814bba-f3fc-4224-9863-ede8f5a186f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:55.142136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9814bba-f3fc-4224-9863-ede8f5a186f6 admin admin] PATH_INFO: `/v3/projects/8cfc5abf8dbc4b2483cec34cd0fe9fc0` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:55.143324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9814bba-f3fc-4224-9863-ede8f5a186f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cfc5abf8dbc4b2483cec34cd0fe9fc0)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:55.145480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9814bba-f3fc-4224-9863-ede8f5a186f6 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:55.202742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b9814bba-f3fc-4224-9863-ede8f5a186f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cfc5abf8dbc4b2483cec34cd0fe9fc0', 'request_id': 'req-b9814bba-f3fc-4224-9863-ede8f5a186f6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:55.210320 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b9814bba-f3fc-4224-9863-ede8f5a186f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cfc5abf8dbc4b2483cec34cd0fe9fc0', 'request_id': 'req-b9814bba-f3fc-4224-9863-ede8f5a186f6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:55.217238 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b9814bba-f3fc-4224-9863-ede8f5a186f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cfc5abf8dbc4b2483cec34cd0fe9fc0', 'request_id': 'req-b9814bba-f3fc-4224-9863-ede8f5a186f6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:55.230635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b9814bba-f3fc-4224-9863-ede8f5a186f6 admin admin] The token cache is being invalidate because project 8cfc5abf8dbc4b2483cec34cd0fe9fc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:55.230984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b9814bba-f3fc-4224-9863-ede8f5a186f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:55.231873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b9814bba-f3fc-4224-9863-ede8f5a186f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:55.233664 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 980/1991] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:55 2022] DELETE /identity/v3/projects/8cfc5abf8dbc4b2483cec34cd0fe9fc0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:55.239740 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfda723-99a8-4551-815e-1e8480545cfe None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:55.240191 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfda723-99a8-4551-815e-1e8480545cfe None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:55.240691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfda723-99a8-4551-815e-1e8480545cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:55.263950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-1cfda723-99a8-4551-815e-1e8480545cfe None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:55.265568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1cfda723-99a8-4551-815e-1e8480545cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:55.320580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:55.357876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1012/1992] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:55.360085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d3234-c04b-44eb-a51a-689505016710 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:55.360613 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d3234-c04b-44eb-a51a-689505016710 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:55.361585 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d3234-c04b-44eb-a51a-689505016710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:55.390783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-417d3234-c04b-44eb-a51a-689505016710 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:55.392111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-417d3234-c04b-44eb-a51a-689505016710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:55.401781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 981/1993] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:55.406686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfda723-99a8-4551-815e-1e8480545cfe None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:55.420479 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfda723-99a8-4551-815e-1e8480545cfe None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:55.426267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51ab3a-ff2c-47f6-8324-cf52396d92c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:55.426965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51ab3a-ff2c-47f6-8324-cf52396d92c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:55.427247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51ab3a-ff2c-47f6-8324-cf52396d92c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:55.427520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51ab3a-ff2c-47f6-8324-cf52396d92c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:55.428331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51ab3a-ff2c-47f6-8324-cf52396d92c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:25:55.429654 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51ab3a-ff2c-47f6-8324-cf52396d92c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:55.432224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51ab3a-ff2c-47f6-8324-cf52396d92c4 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:55.446654 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1013/1994] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:25:55.452530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d3234-c04b-44eb-a51a-689505016710 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:55.465251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d3234-c04b-44eb-a51a-689505016710 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:55.470198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7b3e2-097b-4cac-b05a-7da1d3d73e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:55.470839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7b3e2-097b-4cac-b05a-7da1d3d73e0d admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:55.471155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7b3e2-097b-4cac-b05a-7da1d3d73e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:55.471467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7b3e2-097b-4cac-b05a-7da1d3d73e0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:55.472142 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7b3e2-097b-4cac-b05a-7da1d3d73e0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:55.474167 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7b3e2-097b-4cac-b05a-7da1d3d73e0d admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:55.510323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 982/1995] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:55 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:55.515030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51ab3a-ff2c-47f6-8324-cf52396d92c4 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:55.528922 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51ab3a-ff2c-47f6-8324-cf52396d92c4 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:55.607084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6c8cd-62b7-484d-89bd-1548d29588eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:55.607697 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6c8cd-62b7-484d-89bd-1548d29588eb admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:55.607981 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6c8cd-62b7-484d-89bd-1548d29588eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:55.608258 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6c8cd-62b7-484d-89bd-1548d29588eb admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:55.608897 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6c8cd-62b7-484d-89bd-1548d29588eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:55.610875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6c8cd-62b7-484d-89bd-1548d29588eb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:55.635308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1014/1996] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:55 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:25:55.640555 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7b3e2-097b-4cac-b05a-7da1d3d73e0d admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:55.652663 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7b3e2-097b-4cac-b05a-7da1d3d73e0d admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:55.658801 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300145e-8968-46a2-a1bc-a4ba9dd4ffc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:55.659508 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300145e-8968-46a2-a1bc-a4ba9dd4ffc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:55.660010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300145e-8968-46a2-a1bc-a4ba9dd4ffc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:55.660359 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300145e-8968-46a2-a1bc-a4ba9dd4ffc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:55.661110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300145e-8968-46a2-a1bc-a4ba9dd4ffc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:55.663016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300145e-8968-46a2-a1bc-a4ba9dd4ffc6 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:55.674531 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 983/1997] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:55.678659 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6c8cd-62b7-484d-89bd-1548d29588eb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:55.688957 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6c8cd-62b7-484d-89bd-1548d29588eb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:55.693876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83be18b-b000-4e57-90d9-638cb3bdb311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:55.694486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83be18b-b000-4e57-90d9-638cb3bdb311 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:55.694797 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83be18b-b000-4e57-90d9-638cb3bdb311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:55.695172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83be18b-b000-4e57-90d9-638cb3bdb311 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:55.696305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83be18b-b000-4e57-90d9-638cb3bdb311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:55.697447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83be18b-b000-4e57-90d9-638cb3bdb311 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:55.709897 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1015/1998] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:55.714973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300145e-8968-46a2-a1bc-a4ba9dd4ffc6 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:55.725755 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300145e-8968-46a2-a1bc-a4ba9dd4ffc6 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:55.731423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5eea6-25e2-44ad-843b-e971d67da0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:55.732253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5eea6-25e2-44ad-843b-e971d67da0de admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:55.732571 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5eea6-25e2-44ad-843b-e971d67da0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:55.732898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5eea6-25e2-44ad-843b-e971d67da0de admin admin] PATH_INFO: `/v3/projects/608a3a9070dc4653b79837809915b38f/users/e0a5534bb3824c37bc45dfce4b3faf6f/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:55.752872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5eea6-25e2-44ad-843b-e971d67da0de admin admin] RBAC: Authorizing `identity:create_grant(project_id=608a3a9070dc4653b79837809915b38f, user_id=e0a5534bb3824c37bc45dfce4b3faf6f, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:55.755641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5eea6-25e2-44ad-843b-e971d67da0de admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:55.778566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 984/1999] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:55 2022] PUT /identity/v3/projects/608a3a9070dc4653b79837809915b38f/users/e0a5534bb3824c37bc45dfce4b3faf6f/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:25:55.790115 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773618b-6dee-47ff-ac89-7285bb622e35 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:55.790402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773618b-6dee-47ff-ac89-7285bb622e35 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:55.790841 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773618b-6dee-47ff-ac89-7285bb622e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:55.828921 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-4773618b-6dee-47ff-ac89-7285bb622e35 None None] MFA Rules not processed for user `e0a5534bb3824c37bc45dfce4b3faf6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:55.830214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4773618b-6dee-47ff-ac89-7285bb622e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:55.926928 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1016/2000] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:55.939583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5eea6-25e2-44ad-843b-e971d67da0de admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:55.941428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-1ca5eea6-25e2-44ad-843b-e971d67da0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:55.962267 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:56.009220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773618b-6dee-47ff-ac89-7285bb622e35 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:56.010738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4773618b-6dee-47ff-ac89-7285bb622e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:56.012620 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5eea6-25e2-44ad-843b-e971d67da0de admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:56.016641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55357f0-7281-4413-a0cb-2762078e87e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:56.017188 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55357f0-7281-4413-a0cb-2762078e87e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:56.017474 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55357f0-7281-4413-a0cb-2762078e87e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:56.017748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55357f0-7281-4413-a0cb-2762078e87e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:56.028635 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55357f0-7281-4413-a0cb-2762078e87e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:56.030285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55357f0-7281-4413-a0cb-2762078e87e4 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:56.046455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 985/2001] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:56.066531 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:56.067583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55357f0-7281-4413-a0cb-2762078e87e4 service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:56.092884 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773618b-6dee-47ff-ac89-7285bb622e35 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:56.098024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd151f4-c2eb-4ad3-941f-1c83f6da4cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:56.098756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd151f4-c2eb-4ad3-941f-1c83f6da4cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:56.099033 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd151f4-c2eb-4ad3-941f-1c83f6da4cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:56.099306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd151f4-c2eb-4ad3-941f-1c83f6da4cd8 admin admin] PATH_INFO: `/v3/users/fbd2dc2367a14fe78f7344841b901746` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:56.100814 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd151f4-c2eb-4ad3-941f-1c83f6da4cd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbd2dc2367a14fe78f7344841b901746)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:56.102980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd151f4-c2eb-4ad3-941f-1c83f6da4cd8 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:56.145920 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55357f0-7281-4413-a0cb-2762078e87e4 service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:56.150821 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e0456-e809-4e53-ac29-cc8dc073cbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:56.151514 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e0456-e809-4e53-ac29-cc8dc073cbfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:56.152358 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e0456-e809-4e53-ac29-cc8dc073cbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:56.152764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e0456-e809-4e53-ac29-cc8dc073cbfa admin admin] PATH_INFO: `/v3/users/e0a5534bb3824c37bc45dfce4b3faf6f` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:56.154638 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e0456-e809-4e53-ac29-cc8dc073cbfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0a5534bb3824c37bc45dfce4b3faf6f)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:56.162864 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e0456-e809-4e53-ac29-cc8dc073cbfa admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:56.188226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4cd151f4-c2eb-4ad3-941f-1c83f6da4cd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbd2dc2367a14fe78f7344841b901746'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:56.209005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4cd151f4-c2eb-4ad3-941f-1c83f6da4cd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbd2dc2367a14fe78f7344841b901746'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:56.222073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4cd151f4-c2eb-4ad3-941f-1c83f6da4cd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbd2dc2367a14fe78f7344841b901746'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:56.224355 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4cd151f4-c2eb-4ad3-941f-1c83f6da4cd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbd2dc2367a14fe78f7344841b901746'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:56.231565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-861e0456-e809-4e53-ac29-cc8dc073cbfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0a5534bb3824c37bc45dfce4b3faf6f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:56.233447 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-861e0456-e809-4e53-ac29-cc8dc073cbfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0a5534bb3824c37bc45dfce4b3faf6f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:56.239114 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4cd151f4-c2eb-4ad3-941f-1c83f6da4cd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbd2dc2367a14fe78f7344841b901746'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:56.245358 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-861e0456-e809-4e53-ac29-cc8dc073cbfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0a5534bb3824c37bc45dfce4b3faf6f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:56.247252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1017/2002] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:56 2022] DELETE /identity/v3/users/fbd2dc2367a14fe78f7344841b901746 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:25:56.252567 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06694c93-8c6f-4ddb-a435-82c979078189 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:56.253469 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06694c93-8c6f-4ddb-a435-82c979078189 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:56.253849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06694c93-8c6f-4ddb-a435-82c979078189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:56.270156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-861e0456-e809-4e53-ac29-cc8dc073cbfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0a5534bb3824c37bc45dfce4b3faf6f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:56.285582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-861e0456-e809-4e53-ac29-cc8dc073cbfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0a5534bb3824c37bc45dfce4b3faf6f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:56.287448 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-06694c93-8c6f-4ddb-a435-82c979078189 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:56.289212 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-06694c93-8c6f-4ddb-a435-82c979078189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:56.295407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 986/2003] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:56 2022] DELETE /identity/v3/users/e0a5534bb3824c37bc45dfce4b3faf6f => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:25:56.301389 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668620a6-0521-4064-8abc-bf44224009ae None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:56.302603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668620a6-0521-4064-8abc-bf44224009ae None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:56.303433 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668620a6-0521-4064-8abc-bf44224009ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:56.346554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-668620a6-0521-4064-8abc-bf44224009ae None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:56.348086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-668620a6-0521-4064-8abc-bf44224009ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:56.369216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1018/2004] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:56.381193 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06694c93-8c6f-4ddb-a435-82c979078189 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:56.383806 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-06694c93-8c6f-4ddb-a435-82c979078189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:56.389015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:56.421643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 987/2005] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:56.431243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668620a6-0521-4064-8abc-bf44224009ae None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:56.432931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-668620a6-0521-4064-8abc-bf44224009ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:56.465866 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06694c93-8c6f-4ddb-a435-82c979078189 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:56.486741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:56.499008 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668620a6-0521-4064-8abc-bf44224009ae None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:56.502860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b43ae6-4248-411b-b725-55110cb35e9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:56.503493 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b43ae6-4248-411b-b725-55110cb35e9e service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:56.503809 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b43ae6-4248-411b-b725-55110cb35e9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:56.504118 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b43ae6-4248-411b-b725-55110cb35e9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:56.515696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b43ae6-4248-411b-b725-55110cb35e9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:56.517311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b43ae6-4248-411b-b725-55110cb35e9e service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:56.528621 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 988/2006] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:25:56.532998 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80a008-bd54-41e6-9da5-3dbf15accaa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:56.533649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80a008-bd54-41e6-9da5-3dbf15accaa5 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:56.533931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80a008-bd54-41e6-9da5-3dbf15accaa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:56.534209 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80a008-bd54-41e6-9da5-3dbf15accaa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:56.543956 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80a008-bd54-41e6-9da5-3dbf15accaa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:56.545573 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80a008-bd54-41e6-9da5-3dbf15accaa5 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:56.555979 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1019/2007] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:25:56.975385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b43ae6-4248-411b-b725-55110cb35e9e service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:56.976870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e3b43ae6-4248-411b-b725-55110cb35e9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:56.994414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b43ae6-4248-411b-b725-55110cb35e9e service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:57.002943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d87d34-ab46-48cf-82a3-f197c0a2a7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:57.005910 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d87d34-ab46-48cf-82a3-f197c0a2a7dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:57.006372 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d87d34-ab46-48cf-82a3-f197c0a2a7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:57.011743 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d87d34-ab46-48cf-82a3-f197c0a2a7dd admin admin] PATH_INFO: `/v3/projects/608a3a9070dc4653b79837809915b38f` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:57.013020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80a008-bd54-41e6-9da5-3dbf15accaa5 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:57.014015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d87d34-ab46-48cf-82a3-f197c0a2a7dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=608a3a9070dc4653b79837809915b38f)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:57.014287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-df80a008-bd54-41e6-9da5-3dbf15accaa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:57.016084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d87d34-ab46-48cf-82a3-f197c0a2a7dd admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:57.032553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80a008-bd54-41e6-9da5-3dbf15accaa5 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:57.047232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:57.049855 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:57.050310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:57.052593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8 admin admin] PATH_INFO: `/v3/projects/22c10af3726e4d9189d5c8ba3454c86c` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:57.055487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22c10af3726e4d9189d5c8ba3454c86c)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:57.060163 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:57.095954 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-74d87d34-ab46-48cf-82a3-f197c0a2a7dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '608a3a9070dc4653b79837809915b38f', 'request_id': 'req-74d87d34-ab46-48cf-82a3-f197c0a2a7dd'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:57.103884 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-74d87d34-ab46-48cf-82a3-f197c0a2a7dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '608a3a9070dc4653b79837809915b38f', 'request_id': 'req-74d87d34-ab46-48cf-82a3-f197c0a2a7dd'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:57.112991 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-74d87d34-ab46-48cf-82a3-f197c0a2a7dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '608a3a9070dc4653b79837809915b38f', 'request_id': 'req-74d87d34-ab46-48cf-82a3-f197c0a2a7dd'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:57.129140 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-74d87d34-ab46-48cf-82a3-f197c0a2a7dd admin admin] The token cache is being invalidate because project 608a3a9070dc4653b79837809915b38f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:57.129496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-74d87d34-ab46-48cf-82a3-f197c0a2a7dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:57.130321 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-74d87d34-ab46-48cf-82a3-f197c0a2a7dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:57.142698 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 989/2008] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:56 2022] DELETE /identity/v3/projects/608a3a9070dc4653b79837809915b38f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:25:57.143251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22c10af3726e4d9189d5c8ba3454c86c', 'request_id': 'req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:57.146769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af47918-37b1-43d2-97a2-965a53925739 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:57.147233 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af47918-37b1-43d2-97a2-965a53925739 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:57.147393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af47918-37b1-43d2-97a2-965a53925739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:57.148872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22c10af3726e4d9189d5c8ba3454c86c', 'request_id': 'req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:57.160029 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22c10af3726e4d9189d5c8ba3454c86c', 'request_id': 'req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:57.172270 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-7af47918-37b1-43d2-97a2-965a53925739 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:57.173560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-7af47918-37b1-43d2-97a2-965a53925739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:57.174511 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8 admin admin] The token cache is being invalidate because project 22c10af3726e4d9189d5c8ba3454c86c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:57.174913 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:57.176072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:57.177600 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1020/2009] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:57 2022] DELETE /identity/v3/projects/22c10af3726e4d9189d5c8ba3454c86c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:57.182927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:57.184522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:57.262200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76aef3c-1182-4b25-9c73-f9d67cf2a8b8 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:57.268164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda35f3-305b-4f4a-b7ba-94bd0c7d7e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:57.268905 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda35f3-305b-4f4a-b7ba-94bd0c7d7e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:57.269144 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda35f3-305b-4f4a-b7ba-94bd0c7d7e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:57.269422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda35f3-305b-4f4a-b7ba-94bd0c7d7e5e admin admin] PATH_INFO: `/v3/users/379c68a4eaf94663a713fb4814d818d4` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:57.270808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda35f3-305b-4f4a-b7ba-94bd0c7d7e5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=379c68a4eaf94663a713fb4814d818d4)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:57.272798 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda35f3-305b-4f4a-b7ba-94bd0c7d7e5e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:57.297868 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 990/2010] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:57.300686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31dd08-4448-42cc-9544-bcecc7483328 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:57.301235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31dd08-4448-42cc-9544-bcecc7483328 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:57.301735 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31dd08-4448-42cc-9544-bcecc7483328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:57.315122 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:57.330770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-4c31dd08-4448-42cc-9544-bcecc7483328 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:57.333457 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4c31dd08-4448-42cc-9544-bcecc7483328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:57.343934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 991/2011] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:57.351449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31dd08-4448-42cc-9544-bcecc7483328 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:57.353579 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9eda35f3-305b-4f4a-b7ba-94bd0c7d7e5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '379c68a4eaf94663a713fb4814d818d4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:57.354965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:57.364931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31dd08-4448-42cc-9544-bcecc7483328 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:57.375335 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb48a0a-7c90-43df-86e5-d57dc64f4b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:57.376024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb48a0a-7c90-43df-86e5-d57dc64f4b7f admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:57.376305 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb48a0a-7c90-43df-86e5-d57dc64f4b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:57.376727 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb48a0a-7c90-43df-86e5-d57dc64f4b7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:57.377447 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb48a0a-7c90-43df-86e5-d57dc64f4b7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:25:57.377608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9eda35f3-305b-4f4a-b7ba-94bd0c7d7e5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '379c68a4eaf94663a713fb4814d818d4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:57.377795 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb48a0a-7c90-43df-86e5-d57dc64f4b7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:57.379385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb48a0a-7c90-43df-86e5-d57dc64f4b7f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:57.389911 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9eda35f3-305b-4f4a-b7ba-94bd0c7d7e5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '379c68a4eaf94663a713fb4814d818d4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:57.390561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9eda35f3-305b-4f4a-b7ba-94bd0c7d7e5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '379c68a4eaf94663a713fb4814d818d4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:57.400810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 992/2012] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:25:57.404203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb48a0a-7c90-43df-86e5-d57dc64f4b7f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:57.407803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9eda35f3-305b-4f4a-b7ba-94bd0c7d7e5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '379c68a4eaf94663a713fb4814d818d4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:57.408023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-dcb48a0a-7c90-43df-86e5-d57dc64f4b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:57.419914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1021/2013] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:25:57 2022] DELETE /identity/v3/users/379c68a4eaf94663a713fb4814d818d4 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:57.434000 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb48a0a-7c90-43df-86e5-d57dc64f4b7f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:57.576646 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e45c77-b6a2-42b2-a494-33874c9ee269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:57.578013 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e45c77-b6a2-42b2-a494-33874c9ee269 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:57.578589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e45c77-b6a2-42b2-a494-33874c9ee269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:57.579206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e45c77-b6a2-42b2-a494-33874c9ee269 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:57.581527 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e45c77-b6a2-42b2-a494-33874c9ee269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:57.583826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e45c77-b6a2-42b2-a494-33874c9ee269 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:57.641524 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 993/2014] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:57 2022] POST /identity/v3/projects => generated 361 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:25:57.644820 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda35f3-305b-4f4a-b7ba-94bd0c7d7e5e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:57.663332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda35f3-305b-4f4a-b7ba-94bd0c7d7e5e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:57.779188 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23727e2c-ac67-4cee-a68a-b83a07b418c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:57.779836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23727e2c-ac67-4cee-a68a-b83a07b418c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:57.780120 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23727e2c-ac67-4cee-a68a-b83a07b418c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:57.781136 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23727e2c-ac67-4cee-a68a-b83a07b418c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:57.781847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23727e2c-ac67-4cee-a68a-b83a07b418c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:57.784769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23727e2c-ac67-4cee-a68a-b83a07b418c5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:57.820182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1022/2015] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:57 2022] POST /identity/v3/users => generated 281 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:57.826708 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e45c77-b6a2-42b2-a494-33874c9ee269 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:57.866104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e45c77-b6a2-42b2-a494-33874c9ee269 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:57.880175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0bd83-0670-4d9d-afdb-5489107f3ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:57.880828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0bd83-0670-4d9d-afdb-5489107f3ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:57.881144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0bd83-0670-4d9d-afdb-5489107f3ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:57.881423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0bd83-0670-4d9d-afdb-5489107f3ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:57.882292 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0bd83-0670-4d9d-afdb-5489107f3ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:57.883941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0bd83-0670-4d9d-afdb-5489107f3ae9 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:57.909786 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 994/2016] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:57.915944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23727e2c-ac67-4cee-a68a-b83a07b418c5 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:57.927685 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23727e2c-ac67-4cee-a68a-b83a07b418c5 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:57.934769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe12de-80fe-4bce-9bfe-381d5f2d8086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:57.935507 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe12de-80fe-4bce-9bfe-381d5f2d8086 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:57.935859 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe12de-80fe-4bce-9bfe-381d5f2d8086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:57.936216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe12de-80fe-4bce-9bfe-381d5f2d8086 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:57.936983 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe12de-80fe-4bce-9bfe-381d5f2d8086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:57.938855 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe12de-80fe-4bce-9bfe-381d5f2d8086 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:57.952318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1023/2017] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:57.956717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0bd83-0670-4d9d-afdb-5489107f3ae9 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:57.970819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0bd83-0670-4d9d-afdb-5489107f3ae9 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:57.976588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c42d3d-e925-4cdc-b83f-406e268dfa07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:57.977363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c42d3d-e925-4cdc-b83f-406e268dfa07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:57.977682 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c42d3d-e925-4cdc-b83f-406e268dfa07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:57.978010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c42d3d-e925-4cdc-b83f-406e268dfa07 admin admin] PATH_INFO: `/v3/projects/40b955b963bd44199d476f1451842bde/users/d278d1d334f2424ea82f97229e1224b0/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:57.995067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c42d3d-e925-4cdc-b83f-406e268dfa07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40b955b963bd44199d476f1451842bde, user_id=d278d1d334f2424ea82f97229e1224b0, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:57.999116 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c42d3d-e925-4cdc-b83f-406e268dfa07 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:58.020506 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 995/2018] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:57 2022] PUT /identity/v3/projects/40b955b963bd44199d476f1451842bde/users/d278d1d334f2424ea82f97229e1224b0/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:58.029298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe12de-80fe-4bce-9bfe-381d5f2d8086 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:58.029298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-26fe12de-80fe-4bce-9bfe-381d5f2d8086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:58.032936 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad449de-a5de-42b2-a586-f7a76f2c5d12 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:58.033257 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad449de-a5de-42b2-a586-f7a76f2c5d12 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:58.033482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad449de-a5de-42b2-a586-f7a76f2c5d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:58.043033 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe12de-80fe-4bce-9bfe-381d5f2d8086 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:58.068039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-6ad449de-a5de-42b2-a586-f7a76f2c5d12 None None] MFA Rules not processed for user `d278d1d334f2424ea82f97229e1224b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:58.069299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6ad449de-a5de-42b2-a586-f7a76f2c5d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:58.124988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3ca5a-f312-46f2-bae1-2c22aa9f8d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:58.125678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3ca5a-f312-46f2-bae1-2c22aa9f8d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:58.125990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3ca5a-f312-46f2-bae1-2c22aa9f8d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:58.126220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3ca5a-f312-46f2-bae1-2c22aa9f8d9e admin admin] PATH_INFO: `/v3/projects/090b210e207f4446b1e06cdccc75b1d1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:58.127384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3ca5a-f312-46f2-bae1-2c22aa9f8d9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=090b210e207f4446b1e06cdccc75b1d1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:58.129383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3ca5a-f312-46f2-bae1-2c22aa9f8d9e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:58.148944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 996/2019] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:58.163869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad449de-a5de-42b2-a586-f7a76f2c5d12 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:58.170198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6ad449de-a5de-42b2-a586-f7a76f2c5d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:58.208830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:58.213922 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:58.218033 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-57d3ca5a-f312-46f2-bae1-2c22aa9f8d9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '090b210e207f4446b1e06cdccc75b1d1', 'request_id': 'req-57d3ca5a-f312-46f2-bae1-2c22aa9f8d9e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:58.227989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-57d3ca5a-f312-46f2-bae1-2c22aa9f8d9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '090b210e207f4446b1e06cdccc75b1d1', 'request_id': 'req-57d3ca5a-f312-46f2-bae1-2c22aa9f8d9e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:58.239970 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-57d3ca5a-f312-46f2-bae1-2c22aa9f8d9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '090b210e207f4446b1e06cdccc75b1d1', 'request_id': 'req-57d3ca5a-f312-46f2-bae1-2c22aa9f8d9e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:58.263391 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-57d3ca5a-f312-46f2-bae1-2c22aa9f8d9e admin admin] The token cache is being invalidate because project 090b210e207f4446b1e06cdccc75b1d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:25:58.263703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-57d3ca5a-f312-46f2-bae1-2c22aa9f8d9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:58.264531 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-57d3ca5a-f312-46f2-bae1-2c22aa9f8d9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:25:58.267356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1024/2020] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:25:58 2022] DELETE /identity/v3/projects/090b210e207f4446b1e06cdccc75b1d1 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:25:58.273752 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133f750-f408-4e47-b871-8a1f6de4d4a2 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:58.274104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133f750-f408-4e47-b871-8a1f6de4d4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:58.274104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133f750-f408-4e47-b871-8a1f6de4d4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:58.278143 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad449de-a5de-42b2-a586-f7a76f2c5d12 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:58.298276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9133f750-f408-4e47-b871-8a1f6de4d4a2 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:58.299744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9133f750-f408-4e47-b871-8a1f6de4d4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:58.354431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b178c26-7a70-4609-bfc0-67e340fb7f2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:58.355813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b178c26-7a70-4609-bfc0-67e340fb7f2e service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:58.356299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b178c26-7a70-4609-bfc0-67e340fb7f2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:58.356831 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b178c26-7a70-4609-bfc0-67e340fb7f2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:58.359623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-1b178c26-7a70-4609-bfc0-67e340fb7f2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:58.419747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1025/2021] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:58.435307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94a26d-c8f5-4415-9f59-65018fcb0ab1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:58.436081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94a26d-c8f5-4415-9f59-65018fcb0ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:58.436524 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94a26d-c8f5-4415-9f59-65018fcb0ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:58.466971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fe94a26d-c8f5-4415-9f59-65018fcb0ab1 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:58.467890 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fe94a26d-c8f5-4415-9f59-65018fcb0ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:58.470842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b178c26-7a70-4609-bfc0-67e340fb7f2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:58.472914 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b178c26-7a70-4609-bfc0-67e340fb7f2e service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:58.480512 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1026/2022] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:58.486842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 997/2023] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:25:58.496729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94a26d-c8f5-4415-9f59-65018fcb0ab1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:58.519732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94a26d-c8f5-4415-9f59-65018fcb0ab1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:58.526185 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25532c-b492-4a77-bc62-09cab83ef64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:58.529446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25532c-b492-4a77-bc62-09cab83ef64b admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:58.530244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25532c-b492-4a77-bc62-09cab83ef64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:58.531059 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25532c-b492-4a77-bc62-09cab83ef64b admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:58.533267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25532c-b492-4a77-bc62-09cab83ef64b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:25:58.534750 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25532c-b492-4a77-bc62-09cab83ef64b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:58.538610 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25532c-b492-4a77-bc62-09cab83ef64b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:58.566637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1027/2024] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:25:58.571066 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b178c26-7a70-4609-bfc0-67e340fb7f2e service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:58.583792 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b178c26-7a70-4609-bfc0-67e340fb7f2e service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:58.590842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5a915-c2d2-475c-8147-a340bb4ddba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:58.591465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5a915-c2d2-475c-8147-a340bb4ddba4 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:58.591786 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5a915-c2d2-475c-8147-a340bb4ddba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:58.592014 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5a915-c2d2-475c-8147-a340bb4ddba4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:58.592665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5a915-c2d2-475c-8147-a340bb4ddba4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:58.594965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5a915-c2d2-475c-8147-a340bb4ddba4 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:58.665112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 998/2025] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:25:58 2022] POST /identity/v3/projects => generated 399 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:25:58.667152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25532c-b492-4a77-bc62-09cab83ef64b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:58.686765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25532c-b492-4a77-bc62-09cab83ef64b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:58.795937 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2bcd7-c825-42cd-835f-f7b14c6fa3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:58.797011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2bcd7-c825-42cd-835f-f7b14c6fa3ff admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:58.797550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2bcd7-c825-42cd-835f-f7b14c6fa3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:58.797990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2bcd7-c825-42cd-835f-f7b14c6fa3ff admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:58.798529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2bcd7-c825-42cd-835f-f7b14c6fa3ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:58.801206 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2bcd7-c825-42cd-835f-f7b14c6fa3ff admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:58.851506 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5a915-c2d2-475c-8147-a340bb4ddba4 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:58.859976 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1028/2026] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:25:58 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:25:58.874007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5a915-c2d2-475c-8147-a340bb4ddba4 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:58.881702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdd06e-c83d-4c4f-a5c6-0bacf5399225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:58.883684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdd06e-c83d-4c4f-a5c6-0bacf5399225 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:58.884253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdd06e-c83d-4c4f-a5c6-0bacf5399225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:58.884699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdd06e-c83d-4c4f-a5c6-0bacf5399225 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:58.885630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fdd06e-c83d-4c4f-a5c6-0bacf5399225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:58.887776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fdd06e-c83d-4c4f-a5c6-0bacf5399225 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:58.906632 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 999/2027] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:58.926409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2bcd7-c825-42cd-835f-f7b14c6fa3ff admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:58.981250 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2bcd7-c825-42cd-835f-f7b14c6fa3ff admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:59.009111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b9e8a-a4d5-4809-ba47-0d1cb9c50df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:59.009111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b9e8a-a4d5-4809-ba47-0d1cb9c50df6 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:59.009417 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b9e8a-a4d5-4809-ba47-0d1cb9c50df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:59.010313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b9e8a-a4d5-4809-ba47-0d1cb9c50df6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:59.011075 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b9e8a-a4d5-4809-ba47-0d1cb9c50df6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:59.014202 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b9e8a-a4d5-4809-ba47-0d1cb9c50df6 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:59.065697 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1029/2028] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:59.067705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdd06e-c83d-4c4f-a5c6-0bacf5399225 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:59.107722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdd06e-c83d-4c4f-a5c6-0bacf5399225 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:59.114276 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92670ee9-962e-4843-8675-5e83570a0649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:59.114990 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92670ee9-962e-4843-8675-5e83570a0649 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:59.115278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92670ee9-962e-4843-8675-5e83570a0649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:59.116206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92670ee9-962e-4843-8675-5e83570a0649 admin admin] PATH_INFO: `/v3/projects/e1c19b2533dd4b3bae7648251a6f455a/users/cb44c8d09a6d4826b053526aa2c369ec/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:59.135393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92670ee9-962e-4843-8675-5e83570a0649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c19b2533dd4b3bae7648251a6f455a, user_id=cb44c8d09a6d4826b053526aa2c369ec, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:59.138872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92670ee9-962e-4843-8675-5e83570a0649 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:59.157762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1000/2029] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:25:59 2022] PUT /identity/v3/projects/e1c19b2533dd4b3bae7648251a6f455a/users/cb44c8d09a6d4826b053526aa2c369ec/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:25:59.171079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198802f-6e1f-4614-bd4f-34e85f166c30 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:59.171530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198802f-6e1f-4614-bd4f-34e85f166c30 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:59.171859 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198802f-6e1f-4614-bd4f-34e85f166c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:59.209378 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a198802f-6e1f-4614-bd4f-34e85f166c30 None None] MFA Rules not processed for user `cb44c8d09a6d4826b053526aa2c369ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:59.211408 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a198802f-6e1f-4614-bd4f-34e85f166c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:59.239915 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:59.292053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1030/2030] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:25:59 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:25:59.305988 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92670ee9-962e-4843-8675-5e83570a0649 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:59.309235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-92670ee9-962e-4843-8675-5e83570a0649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:59.364357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:59.390897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92670ee9-962e-4843-8675-5e83570a0649 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:59.396968 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd5e3b-bc80-448f-ad35-4908a6b965d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:59.398073 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd5e3b-bc80-448f-ad35-4908a6b965d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:59.398568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd5e3b-bc80-448f-ad35-4908a6b965d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:59.399069 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd5e3b-bc80-448f-ad35-4908a6b965d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:59.416055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd5e3b-bc80-448f-ad35-4908a6b965d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:59.418890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd5e3b-bc80-448f-ad35-4908a6b965d2 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:59.433000 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1001/2031] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:59 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:26:07.981701 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198802f-6e1f-4614-bd4f-34e85f166c30 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:08.110445 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198802f-6e1f-4614-bd4f-34e85f166c30 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:08.115417 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c25af-3201-41de-a40d-7280df65edbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:08.116070 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c25af-3201-41de-a40d-7280df65edbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:08.116455 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c25af-3201-41de-a40d-7280df65edbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:08.116824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c25af-3201-41de-a40d-7280df65edbe admin admin] PATH_INFO: `/v3/users/cb44c8d09a6d4826b053526aa2c369ec` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:08.118283 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c25af-3201-41de-a40d-7280df65edbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb44c8d09a6d4826b053526aa2c369ec)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:08.120618 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c25af-3201-41de-a40d-7280df65edbe admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:08.209834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-da2c25af-3201-41de-a40d-7280df65edbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb44c8d09a6d4826b053526aa2c369ec'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:08.233650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-da2c25af-3201-41de-a40d-7280df65edbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb44c8d09a6d4826b053526aa2c369ec'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:08.251359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-da2c25af-3201-41de-a40d-7280df65edbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb44c8d09a6d4826b053526aa2c369ec'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:08.252549 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-da2c25af-3201-41de-a40d-7280df65edbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb44c8d09a6d4826b053526aa2c369ec'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:08.271446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-da2c25af-3201-41de-a40d-7280df65edbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb44c8d09a6d4826b053526aa2c369ec'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:08.283290 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1031/2032] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:26:07 2022] DELETE /identity/v3/users/cb44c8d09a6d4826b053526aa2c369ec => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:26:08.288689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8445af9-2150-44bf-b404-ab68970dde74 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:08.289456 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8445af9-2150-44bf-b404-ab68970dde74 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:08.289608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8445af9-2150-44bf-b404-ab68970dde74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:08.322704 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-c8445af9-2150-44bf-b404-ab68970dde74 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:08.325973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c8445af9-2150-44bf-b404-ab68970dde74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:08.451029 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1002/2033] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:08.467590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c25af-3201-41de-a40d-7280df65edbe admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:08.469774 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-da2c25af-3201-41de-a40d-7280df65edbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:08.559256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:08.583810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c25af-3201-41de-a40d-7280df65edbe admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:08.594986 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ef38f-8ef7-47a6-b0a5-7aec74ff144b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:08.596073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ef38f-8ef7-47a6-b0a5-7aec74ff144b service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:08.597370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ef38f-8ef7-47a6-b0a5-7aec74ff144b service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:08.597873 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ef38f-8ef7-47a6-b0a5-7aec74ff144b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:08.617229 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ef38f-8ef7-47a6-b0a5-7aec74ff144b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:08.620830 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ef38f-8ef7-47a6-b0a5-7aec74ff144b service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:08.647676 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1032/2034] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:26:09.187491 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8445af9-2150-44bf-b404-ab68970dde74 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:09.189255 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c8445af9-2150-44bf-b404-ab68970dde74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:09.205986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8445af9-2150-44bf-b404-ab68970dde74 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:09.210867 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a88dae-e883-4aa2-9b5b-14663c4deb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:09.211637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a88dae-e883-4aa2-9b5b-14663c4deb85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:09.211916 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a88dae-e883-4aa2-9b5b-14663c4deb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:09.212183 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a88dae-e883-4aa2-9b5b-14663c4deb85 admin admin] PATH_INFO: `/v3/projects/e1c19b2533dd4b3bae7648251a6f455a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:09.214841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a88dae-e883-4aa2-9b5b-14663c4deb85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1c19b2533dd4b3bae7648251a6f455a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:09.217093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a88dae-e883-4aa2-9b5b-14663c4deb85 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:09.274698 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d5a88dae-e883-4aa2-9b5b-14663c4deb85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1c19b2533dd4b3bae7648251a6f455a', 'request_id': 'req-d5a88dae-e883-4aa2-9b5b-14663c4deb85'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:09.287827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d5a88dae-e883-4aa2-9b5b-14663c4deb85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1c19b2533dd4b3bae7648251a6f455a', 'request_id': 'req-d5a88dae-e883-4aa2-9b5b-14663c4deb85'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:09.298932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d5a88dae-e883-4aa2-9b5b-14663c4deb85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1c19b2533dd4b3bae7648251a6f455a', 'request_id': 'req-d5a88dae-e883-4aa2-9b5b-14663c4deb85'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:09.318162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d5a88dae-e883-4aa2-9b5b-14663c4deb85 admin admin] The token cache is being invalidate because project e1c19b2533dd4b3bae7648251a6f455a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:26:09.318582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d5a88dae-e883-4aa2-9b5b-14663c4deb85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:09.319648 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d5a88dae-e883-4aa2-9b5b-14663c4deb85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:09.321338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1003/2035] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:26:09 2022] DELETE /identity/v3/projects/e1c19b2533dd4b3bae7648251a6f455a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:26:09.330990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92e3e4-6025-413a-907e-37bd178c11ed None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:09.331614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92e3e4-6025-413a-907e-37bd178c11ed None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:09.332014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92e3e4-6025-413a-907e-37bd178c11ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:09.364006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-be92e3e4-6025-413a-907e-37bd178c11ed None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:09.365984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-be92e3e4-6025-413a-907e-37bd178c11ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:09.469993 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1033/2036] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:09.477917 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f035fac-1aa3-46a3-903a-7d6cb58fb646 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:09.478367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f035fac-1aa3-46a3-903a-7d6cb58fb646 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:09.478735 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f035fac-1aa3-46a3-903a-7d6cb58fb646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:09.503393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-2f035fac-1aa3-46a3-903a-7d6cb58fb646 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:09.504720 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-2f035fac-1aa3-46a3-903a-7d6cb58fb646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:09.516698 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1004/2037] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:09.519010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92e3e4-6025-413a-907e-37bd178c11ed None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:09.538513 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92e3e4-6025-413a-907e-37bd178c11ed None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:09.547451 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191c37b-edf0-4403-b9df-85ac5627fe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:09.548336 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191c37b-edf0-4403-b9df-85ac5627fe90 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:09.548665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191c37b-edf0-4403-b9df-85ac5627fe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:09.548938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191c37b-edf0-4403-b9df-85ac5627fe90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:09.549980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191c37b-edf0-4403-b9df-85ac5627fe90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:26:09.550528 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191c37b-edf0-4403-b9df-85ac5627fe90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:09.553635 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191c37b-edf0-4403-b9df-85ac5627fe90 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:09.576283 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1034/2038] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:26:09.581269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f035fac-1aa3-46a3-903a-7d6cb58fb646 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:09.593324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f035fac-1aa3-46a3-903a-7d6cb58fb646 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:09.598376 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e2b48-e6be-4501-9b25-165f73a6e362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:09.599058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e2b48-e6be-4501-9b25-165f73a6e362 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:09.599337 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e2b48-e6be-4501-9b25-165f73a6e362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:09.599640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e2b48-e6be-4501-9b25-165f73a6e362 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:09.600305 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e2b48-e6be-4501-9b25-165f73a6e362 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:09.602770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e2b48-e6be-4501-9b25-165f73a6e362 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:09.641424 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1005/2039] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:26:09 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:26:09.645691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191c37b-edf0-4403-b9df-85ac5627fe90 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:09.658511 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191c37b-edf0-4403-b9df-85ac5627fe90 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:09.732350 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50f32c-7479-4a03-bbfe-91c13e89bb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:09.732895 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50f32c-7479-4a03-bbfe-91c13e89bb13 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:09.733229 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50f32c-7479-4a03-bbfe-91c13e89bb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:09.733505 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50f32c-7479-4a03-bbfe-91c13e89bb13 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:09.734156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50f32c-7479-4a03-bbfe-91c13e89bb13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:09.736245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50f32c-7479-4a03-bbfe-91c13e89bb13 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:09.762744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1035/2040] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:26:09 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:26:09.767285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e2b48-e6be-4501-9b25-165f73a6e362 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:09.782764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e2b48-e6be-4501-9b25-165f73a6e362 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:09.790636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3259ca-a9e7-4f1b-a8f2-0d0284526bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:09.791295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3259ca-a9e7-4f1b-a8f2-0d0284526bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:09.791578 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3259ca-a9e7-4f1b-a8f2-0d0284526bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:09.791858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3259ca-a9e7-4f1b-a8f2-0d0284526bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:09.793354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3259ca-a9e7-4f1b-a8f2-0d0284526bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:09.795135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3259ca-a9e7-4f1b-a8f2-0d0284526bc6 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:09.811549 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1006/2041] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:26:09.812890 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50f32c-7479-4a03-bbfe-91c13e89bb13 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:09.824209 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50f32c-7479-4a03-bbfe-91c13e89bb13 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:09.829021 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62598515-ffb3-4d70-98b5-1975f6bd6bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:09.829663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62598515-ffb3-4d70-98b5-1975f6bd6bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:09.829940 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62598515-ffb3-4d70-98b5-1975f6bd6bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:09.830208 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62598515-ffb3-4d70-98b5-1975f6bd6bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:09.831028 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62598515-ffb3-4d70-98b5-1975f6bd6bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:09.832641 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62598515-ffb3-4d70-98b5-1975f6bd6bb5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:09.849413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1036/2042] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:26:09.853886 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3259ca-a9e7-4f1b-a8f2-0d0284526bc6 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:09.865369 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3259ca-a9e7-4f1b-a8f2-0d0284526bc6 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:09.875431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a313d-ebed-4be9-b201-1dfbe833cfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:09.876105 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a313d-ebed-4be9-b201-1dfbe833cfb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:09.876385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a313d-ebed-4be9-b201-1dfbe833cfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:09.876661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a313d-ebed-4be9-b201-1dfbe833cfb1 admin admin] PATH_INFO: `/v3/projects/2cfc99aa2ea246fb845d429885ff5209/users/eb81364013af425e9956dd09ceb150e6/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:09.895342 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a313d-ebed-4be9-b201-1dfbe833cfb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cfc99aa2ea246fb845d429885ff5209, user_id=eb81364013af425e9956dd09ceb150e6, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:09.898993 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a313d-ebed-4be9-b201-1dfbe833cfb1 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:09.909892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:09.914987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1007/2043] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:26:09 2022] PUT /identity/v3/projects/2cfc99aa2ea246fb845d429885ff5209/users/eb81364013af425e9956dd09ceb150e6/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:26:09.922630 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c28bde-1354-452b-abea-5878841bfe53 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:09.923256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c28bde-1354-452b-abea-5878841bfe53 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:09.923587 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c28bde-1354-452b-abea-5878841bfe53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:09.953625 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c2c28bde-1354-452b-abea-5878841bfe53 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:09.955148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c2c28bde-1354-452b-abea-5878841bfe53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:10.032833 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:10.064152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1037/2044] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:10.076904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a313d-ebed-4be9-b201-1dfbe833cfb1 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:10.081180 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3f9a313d-ebed-4be9-b201-1dfbe833cfb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:10.186268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a313d-ebed-4be9-b201-1dfbe833cfb1 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:10.192434 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8fcc3-276a-4b3e-8b1d-214c44e8d2e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:10.193463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8fcc3-276a-4b3e-8b1d-214c44e8d2e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:10.193936 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8fcc3-276a-4b3e-8b1d-214c44e8d2e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:10.194429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8fcc3-276a-4b3e-8b1d-214c44e8d2e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:10.212128 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8fcc3-276a-4b3e-8b1d-214c44e8d2e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:10.216407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8fcc3-276a-4b3e-8b1d-214c44e8d2e6 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:10.233567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1008/2045] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:26:16.815854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5ac8b-ac85-4e30-b8c4-5f5a69b03c0e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:16.816427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5ac8b-ac85-4e30-b8c4-5f5a69b03c0e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:16.816427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5ac8b-ac85-4e30-b8c4-5f5a69b03c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:16.880026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-04f5ac8b-ac85-4e30-b8c4-5f5a69b03c0e None None] MFA Rules not processed for user `ba49197c9a3446b68bc92c48abf57b37`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:16.882384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-04f5ac8b-ac85-4e30-b8c4-5f5a69b03c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:17.060043 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1038/2046] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:26:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:17.737111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b72b0a-fe9e-44f7-807e-dab9901124d5 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:17.737611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b72b0a-fe9e-44f7-807e-dab9901124d5 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:17.737777 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b72b0a-fe9e-44f7-807e-dab9901124d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:17.787870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-59b72b0a-fe9e-44f7-807e-dab9901124d5 None None] MFA Rules not processed for user `eb81364013af425e9956dd09ceb150e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:17.790074 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-59b72b0a-fe9e-44f7-807e-dab9901124d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:17.991087 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1009/2047] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:26:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:17.998749 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5ac8b-ac85-4e30-b8c4-5f5a69b03c0e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:18.075835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5ac8b-ac85-4e30-b8c4-5f5a69b03c0e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:18.101607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18076f1a-1a89-44a9-b2b7-485e6aaf9f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:18.102249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18076f1a-1a89-44a9-b2b7-485e6aaf9f42 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:18.108140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18076f1a-1a89-44a9-b2b7-485e6aaf9f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:18.108749 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18076f1a-1a89-44a9-b2b7-485e6aaf9f42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:18.110336 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18076f1a-1a89-44a9-b2b7-485e6aaf9f42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:18.113484 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18076f1a-1a89-44a9-b2b7-485e6aaf9f42 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:18.179613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1039/2048] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:26:17 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:26:18.183685 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b72b0a-fe9e-44f7-807e-dab9901124d5 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:18.218584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b72b0a-fe9e-44f7-807e-dab9901124d5 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:18.433769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d539d4-d42f-4c15-a22c-ca8f22aeb5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:18.435082 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d539d4-d42f-4c15-a22c-ca8f22aeb5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:18.435082 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d539d4-d42f-4c15-a22c-ca8f22aeb5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:18.435411 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d539d4-d42f-4c15-a22c-ca8f22aeb5c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:18.436450 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d539d4-d42f-4c15-a22c-ca8f22aeb5c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:18.441328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d539d4-d42f-4c15-a22c-ca8f22aeb5c0 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:18.535256 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1010/2049] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:26:18 2022] POST /identity/v3/users => generated 299 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:26:18.540344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18076f1a-1a89-44a9-b2b7-485e6aaf9f42 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:18.565524 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18076f1a-1a89-44a9-b2b7-485e6aaf9f42 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:18.593899 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0efb0-6133-40cd-b3e6-fd631888b795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:18.594957 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0efb0-6133-40cd-b3e6-fd631888b795 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:18.595453 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0efb0-6133-40cd-b3e6-fd631888b795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:18.596389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0efb0-6133-40cd-b3e6-fd631888b795 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:18.605563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0efb0-6133-40cd-b3e6-fd631888b795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:18.614721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0efb0-6133-40cd-b3e6-fd631888b795 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:18.649467 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1040/2050] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:26:18.651349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d539d4-d42f-4c15-a22c-ca8f22aeb5c0 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:18.669581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d539d4-d42f-4c15-a22c-ca8f22aeb5c0 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:18.685815 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0867b-41b5-43e0-b11f-539e25ff07f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:18.686511 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0867b-41b5-43e0-b11f-539e25ff07f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:18.686797 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0867b-41b5-43e0-b11f-539e25ff07f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:18.687078 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0867b-41b5-43e0-b11f-539e25ff07f0 admin admin] PATH_INFO: `/v3/projects/e1969acf59134ccca626d7cb1015a184/users/5c452a85ac484f89b2f1d6fa8fa55f6e/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:18.727455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef0867b-41b5-43e0-b11f-539e25ff07f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1969acf59134ccca626d7cb1015a184, user_id=5c452a85ac484f89b2f1d6fa8fa55f6e, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:18.738159 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef0867b-41b5-43e0-b11f-539e25ff07f0 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:18.760539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:18.782530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1011/2051] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:26:18 2022] PUT /identity/v3/projects/e1969acf59134ccca626d7cb1015a184/users/5c452a85ac484f89b2f1d6fa8fa55f6e/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:26:18.786881 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0efb0-6133-40cd-b3e6-fd631888b795 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:18.802028 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0efb0-6133-40cd-b3e6-fd631888b795 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:18.934123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe141fc0-24cd-44fe-8646-6102651caa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:18.934266 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe141fc0-24cd-44fe-8646-6102651caa59 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:18.934715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe141fc0-24cd-44fe-8646-6102651caa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:18.935041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe141fc0-24cd-44fe-8646-6102651caa59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:18.935761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe141fc0-24cd-44fe-8646-6102651caa59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:18.939390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe141fc0-24cd-44fe-8646-6102651caa59 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:18.953654 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1041/2052] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:26:18.960242 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0867b-41b5-43e0-b11f-539e25ff07f0 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:18.980316 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0867b-41b5-43e0-b11f-539e25ff07f0 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:18.991611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000053e4-ab5e-4574-969e-2167f63e58ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:18.992551 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000053e4-ab5e-4574-969e-2167f63e58ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:18.993908 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000053e4-ab5e-4574-969e-2167f63e58ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:18.994579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000053e4-ab5e-4574-969e-2167f63e58ca admin admin] PATH_INFO: `/v3/projects/e1969acf59134ccca626d7cb1015a184/users/5c452a85ac484f89b2f1d6fa8fa55f6e/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:18.998390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000053e4-ab5e-4574-969e-2167f63e58ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1969acf59134ccca626d7cb1015a184, user_id=5c452a85ac484f89b2f1d6fa8fa55f6e, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:19.003045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000053e4-ab5e-4574-969e-2167f63e58ca admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:19.028322 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1012/2053] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:26:18 2022] PUT /identity/v3/projects/e1969acf59134ccca626d7cb1015a184/users/5c452a85ac484f89b2f1d6fa8fa55f6e/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:26:25.183603 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a31b01-1999-4531-b7fa-71a2595f9c98 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:25.184248 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a31b01-1999-4531-b7fa-71a2595f9c98 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:25.184248 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a31b01-1999-4531-b7fa-71a2595f9c98 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:25.186578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1042/2054] 10.222.0.88 () {54 vars in 921 bytes} [Tue Aug 23 22:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:26:25.231288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3e158-76d5-40a8-838c-f1a829f2fd09 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:25.232873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3e158-76d5-40a8-838c-f1a829f2fd09 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:25.233372 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3e158-76d5-40a8-838c-f1a829f2fd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:25.285961 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-91b3e158-76d5-40a8-838c-f1a829f2fd09 None None] MFA Rules not processed for user `e642dd7a07ca4afaa40c2769299d8bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:25.290394 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-91b3e158-76d5-40a8-838c-f1a829f2fd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:25.492055 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a31b01-1999-4531-b7fa-71a2595f9c98 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:25.496173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c8a31b01-1999-4531-b7fa-71a2595f9c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:25.550620 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1013/2055] 10.222.0.88 () {58 vars in 1018 bytes} [Tue Aug 23 22:26:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:26:25.571464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3e158-76d5-40a8-838c-f1a829f2fd09 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:25.573648 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-91b3e158-76d5-40a8-838c-f1a829f2fd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:25.591314 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3e158-76d5-40a8-838c-f1a829f2fd09 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:25.599292 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac818385-00f4-4176-819a-daa02fd38ec3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e642dd7a07ca4afaa40c2769299d8bad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:25.600431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac818385-00f4-4176-819a-daa02fd38ec3 service placement] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:25.600919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac818385-00f4-4176-819a-daa02fd38ec3 service placement] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:25.602211 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac818385-00f4-4176-819a-daa02fd38ec3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:25.613825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac818385-00f4-4176-819a-daa02fd38ec3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:25.616752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac818385-00f4-4176-819a-daa02fd38ec3 service placement] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:25.640695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1014/2056] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:26:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:26:25.695434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a31b01-1999-4531-b7fa-71a2595f9c98 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:25.704213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92036ce1-2989-4ee1-8291-dd786df87921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:25.705452 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92036ce1-2989-4ee1-8291-dd786df87921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:25.706299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92036ce1-2989-4ee1-8291-dd786df87921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:25.706892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92036ce1-2989-4ee1-8291-dd786df87921 admin admin] PATH_INFO: `/v3/users/4e884c4f24db45a786618fc28d19bced` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:25.709003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92036ce1-2989-4ee1-8291-dd786df87921 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e884c4f24db45a786618fc28d19bced)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:25.712868 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92036ce1-2989-4ee1-8291-dd786df87921 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:25.718014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:25.796471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-92036ce1-2989-4ee1-8291-dd786df87921 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e884c4f24db45a786618fc28d19bced'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:25.815255 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-92036ce1-2989-4ee1-8291-dd786df87921 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e884c4f24db45a786618fc28d19bced'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:25.829389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-92036ce1-2989-4ee1-8291-dd786df87921 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e884c4f24db45a786618fc28d19bced'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:25.833256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-92036ce1-2989-4ee1-8291-dd786df87921 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e884c4f24db45a786618fc28d19bced'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:25.863778 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-92036ce1-2989-4ee1-8291-dd786df87921 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e884c4f24db45a786618fc28d19bced'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:25.882629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1043/2057] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:26:25 2022] DELETE /identity/v3/users/4e884c4f24db45a786618fc28d19bced => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:26:26.356616 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac818385-00f4-4176-819a-daa02fd38ec3 service placement] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:26.368487 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac818385-00f4-4176-819a-daa02fd38ec3 service placement] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:26.477984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f1e63-6e0f-4588-95f7-7d6c6b4291af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:26.478647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f1e63-6e0f-4588-95f7-7d6c6b4291af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:26.478925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f1e63-6e0f-4588-95f7-7d6c6b4291af admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:26.479236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f1e63-6e0f-4588-95f7-7d6c6b4291af admin admin] PATH_INFO: `/v3/projects/398769c58d0d412aa45024c937820ba1` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:26.480451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f1e63-6e0f-4588-95f7-7d6c6b4291af admin admin] RBAC: Authorizing `identity:delete_project(project_id=398769c58d0d412aa45024c937820ba1)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:26.483152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f1e63-6e0f-4588-95f7-7d6c6b4291af admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:26.554224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-540f1e63-6e0f-4588-95f7-7d6c6b4291af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '398769c58d0d412aa45024c937820ba1', 'request_id': 'req-540f1e63-6e0f-4588-95f7-7d6c6b4291af'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:26.564553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-540f1e63-6e0f-4588-95f7-7d6c6b4291af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '398769c58d0d412aa45024c937820ba1', 'request_id': 'req-540f1e63-6e0f-4588-95f7-7d6c6b4291af'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:26.575789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-540f1e63-6e0f-4588-95f7-7d6c6b4291af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '398769c58d0d412aa45024c937820ba1', 'request_id': 'req-540f1e63-6e0f-4588-95f7-7d6c6b4291af'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:26.588706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:26.595419 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-540f1e63-6e0f-4588-95f7-7d6c6b4291af admin admin] The token cache is being invalidate because project 398769c58d0d412aa45024c937820ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:26:26.597533 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-540f1e63-6e0f-4588-95f7-7d6c6b4291af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:26.598646 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-540f1e63-6e0f-4588-95f7-7d6c6b4291af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:26.607019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1015/2058] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:26:26 2022] DELETE /identity/v3/projects/398769c58d0d412aa45024c937820ba1 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:26:26.608452 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1bd7d-7aa4-4d00-8af5-3eba3085b2b0 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:26.609015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1bd7d-7aa4-4d00-8af5-3eba3085b2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:26.609015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1bd7d-7aa4-4d00-8af5-3eba3085b2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:26.638949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c7f1bd7d-7aa4-4d00-8af5-3eba3085b2b0 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:26.641677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c7f1bd7d-7aa4-4d00-8af5-3eba3085b2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:26.773591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1044/2059] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:26.777357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39d6bf-898d-43b6-97a1-a58fa3efe260 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:26.778997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39d6bf-898d-43b6-97a1-a58fa3efe260 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:26.779293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39d6bf-898d-43b6-97a1-a58fa3efe260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:26.815369 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-ce39d6bf-898d-43b6-97a1-a58fa3efe260 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:26.817512 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ce39d6bf-898d-43b6-97a1-a58fa3efe260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:26.826851 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1016/2060] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:26.831316 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1bd7d-7aa4-4d00-8af5-3eba3085b2b0 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:26.842999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1bd7d-7aa4-4d00-8af5-3eba3085b2b0 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:26.851460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c91c0e-976f-460e-b900-f93c70a33df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:26.851970 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c91c0e-976f-460e-b900-f93c70a33df6 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:26.852295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c91c0e-976f-460e-b900-f93c70a33df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:26.852632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c91c0e-976f-460e-b900-f93c70a33df6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:26.853262 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c91c0e-976f-460e-b900-f93c70a33df6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:26:26.853688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c91c0e-976f-460e-b900-f93c70a33df6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:26.855352 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c91c0e-976f-460e-b900-f93c70a33df6 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:26.873352 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1045/2061] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:26:26.877987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39d6bf-898d-43b6-97a1-a58fa3efe260 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:26.890378 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39d6bf-898d-43b6-97a1-a58fa3efe260 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:26.896302 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b670d15-0697-4f21-b23e-791bfdff5b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:26.897030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b670d15-0697-4f21-b23e-791bfdff5b5d admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:26.897350 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b670d15-0697-4f21-b23e-791bfdff5b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:26.897673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b670d15-0697-4f21-b23e-791bfdff5b5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:26.898331 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b670d15-0697-4f21-b23e-791bfdff5b5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:26.900594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b670d15-0697-4f21-b23e-791bfdff5b5d admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:26.953517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1017/2062] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:26:26 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:26:26.969869 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c91c0e-976f-460e-b900-f93c70a33df6 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:26.984768 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c91c0e-976f-460e-b900-f93c70a33df6 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:27.077317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3815190-aaa8-4bec-9a0e-1dd0b6737b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:27.081811 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3815190-aaa8-4bec-9a0e-1dd0b6737b63 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:27.082304 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3815190-aaa8-4bec-9a0e-1dd0b6737b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:27.082732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3815190-aaa8-4bec-9a0e-1dd0b6737b63 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:27.084658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3815190-aaa8-4bec-9a0e-1dd0b6737b63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:27.089908 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3815190-aaa8-4bec-9a0e-1dd0b6737b63 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:27.108818 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:27.129319 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b670d15-0697-4f21-b23e-791bfdff5b5d admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:27.130205 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1046/2063] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:26:26 2022] POST /identity/v3/users => generated 287 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:26:27.143097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b670d15-0697-4f21-b23e-791bfdff5b5d admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:27.153374 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66844806-2f14-4af1-881a-948d2564b97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:27.154064 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66844806-2f14-4af1-881a-948d2564b97b admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:27.154422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66844806-2f14-4af1-881a-948d2564b97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:27.154768 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66844806-2f14-4af1-881a-948d2564b97b admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:27.156554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66844806-2f14-4af1-881a-948d2564b97b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:27.158790 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66844806-2f14-4af1-881a-948d2564b97b admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:27.172464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1018/2064] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:26:27.178232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3815190-aaa8-4bec-9a0e-1dd0b6737b63 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:27.192881 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3815190-aaa8-4bec-9a0e-1dd0b6737b63 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:27.240030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210cb58-2933-4586-b582-aa02ebcaaa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:27.240835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210cb58-2933-4586-b582-aa02ebcaaa3f admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:27.241505 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210cb58-2933-4586-b582-aa02ebcaaa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:27.241992 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210cb58-2933-4586-b582-aa02ebcaaa3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:27.248263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210cb58-2933-4586-b582-aa02ebcaaa3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:27.250981 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210cb58-2933-4586-b582-aa02ebcaaa3f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:27.286489 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1047/2065] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:26:27.287879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66844806-2f14-4af1-881a-948d2564b97b admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:27.313574 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66844806-2f14-4af1-881a-948d2564b97b admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:27.319134 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b639666-547a-4f1c-8eff-67c17f5d0f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:27.320097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b639666-547a-4f1c-8eff-67c17f5d0f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:27.320482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b639666-547a-4f1c-8eff-67c17f5d0f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:27.320892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b639666-547a-4f1c-8eff-67c17f5d0f0a admin admin] PATH_INFO: `/v3/projects/2dfca0522378453093b94d29d617fc73/users/0b5d58cdcf284732a838a7b04e9c2a51/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:27.339980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b639666-547a-4f1c-8eff-67c17f5d0f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dfca0522378453093b94d29d617fc73, user_id=0b5d58cdcf284732a838a7b04e9c2a51, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:27.343070 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b639666-547a-4f1c-8eff-67c17f5d0f0a admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:27.363152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1019/2066] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:26:27 2022] PUT /identity/v3/projects/2dfca0522378453093b94d29d617fc73/users/0b5d58cdcf284732a838a7b04e9c2a51/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:26:27.376028 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bab56-4e46-4243-9c86-53cbaf4ad5d4 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:27.376379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bab56-4e46-4243-9c86-53cbaf4ad5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:27.376626 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bab56-4e46-4243-9c86-53cbaf4ad5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:27.410593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6a7bab56-4e46-4243-9c86-53cbaf4ad5d4 None None] MFA Rules not processed for user `0b5d58cdcf284732a838a7b04e9c2a51`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:27.411885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6a7bab56-4e46-4243-9c86-53cbaf4ad5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:27.514850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1048/2067] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:26:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:27.519362 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b639666-547a-4f1c-8eff-67c17f5d0f0a admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:27.651186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b639666-547a-4f1c-8eff-67c17f5d0f0a admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:27.658640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75ec70-cbaa-4bb0-9da0-97697646c368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:27.659632 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75ec70-cbaa-4bb0-9da0-97697646c368 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:27.660165 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75ec70-cbaa-4bb0-9da0-97697646c368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:27.660510 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75ec70-cbaa-4bb0-9da0-97697646c368 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:27.661262 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75ec70-cbaa-4bb0-9da0-97697646c368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:27.670002 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75ec70-cbaa-4bb0-9da0-97697646c368 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:27.746172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bab56-4e46-4243-9c86-53cbaf4ad5d4 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:27.753830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1020/2068] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:26:27 2022] POST /identity/v3/projects => generated 373 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:26:27.768447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bab56-4e46-4243-9c86-53cbaf4ad5d4 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:27.850680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198330e-cd33-4ad3-b959-fa9883da746b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:27.851313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198330e-cd33-4ad3-b959-fa9883da746b admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:27.852828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198330e-cd33-4ad3-b959-fa9883da746b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:27.852828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198330e-cd33-4ad3-b959-fa9883da746b admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:27.852828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198330e-cd33-4ad3-b959-fa9883da746b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:27.855277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198330e-cd33-4ad3-b959-fa9883da746b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:27.887098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1049/2069] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:26:27 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:26:27.890803 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75ec70-cbaa-4bb0-9da0-97697646c368 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:27.908901 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75ec70-cbaa-4bb0-9da0-97697646c368 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:27.937937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a5a11-8212-4ba9-9b17-f3abf4df244c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:27.938597 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a5a11-8212-4ba9-9b17-f3abf4df244c admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:27.938883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a5a11-8212-4ba9-9b17-f3abf4df244c admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:27.939165 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a5a11-8212-4ba9-9b17-f3abf4df244c admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:27.939863 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a5a11-8212-4ba9-9b17-f3abf4df244c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:27.941682 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a5a11-8212-4ba9-9b17-f3abf4df244c admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:27.959637 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1021/2070] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:26:27.960776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198330e-cd33-4ad3-b959-fa9883da746b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:27.977098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198330e-cd33-4ad3-b959-fa9883da746b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:27.991897 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb74b52-498f-4da3-b748-ad800ca71cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:27.992627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb74b52-498f-4da3-b748-ad800ca71cef admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:27.992914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb74b52-498f-4da3-b748-ad800ca71cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:28.010944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb74b52-498f-4da3-b748-ad800ca71cef admin admin] PATH_INFO: `/v3/projects/2c85a4108cff482ea9c503e0e2ffb0ab/users/1168299020094117ac5fef393f1878ee/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:28.023299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9843dc6-06b5-43e4-9b65-26f9cb17db4b None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:28.023613 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9843dc6-06b5-43e4-9b65-26f9cb17db4b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:28.023922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9843dc6-06b5-43e4-9b65-26f9cb17db4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:28.060659 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb74b52-498f-4da3-b748-ad800ca71cef admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c85a4108cff482ea9c503e0e2ffb0ab, user_id=1168299020094117ac5fef393f1878ee, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:28.067294 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb74b52-498f-4da3-b748-ad800ca71cef admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:28.116588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-f9843dc6-06b5-43e4-9b65-26f9cb17db4b None None] MFA Rules not processed for user `5c452a85ac484f89b2f1d6fa8fa55f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:28.116588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f9843dc6-06b5-43e4-9b65-26f9cb17db4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:28.152370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1050/2071] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:26:27 2022] PUT /identity/v3/projects/2c85a4108cff482ea9c503e0e2ffb0ab/users/1168299020094117ac5fef393f1878ee/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:26:28.154510 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb74b52-498f-4da3-b748-ad800ca71cef admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:28.210939 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:28.232155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:28.233460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1022/2072] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:26:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:26:28.254124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9843dc6-06b5-43e4-9b65-26f9cb17db4b None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:28.261915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f9843dc6-06b5-43e4-9b65-26f9cb17db4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:28.277382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb74b52-498f-4da3-b748-ad800ca71cef admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:28.284157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed3057-5fef-499f-a2cb-e38a0fee97f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:28.285332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed3057-5fef-499f-a2cb-e38a0fee97f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:28.286414 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed3057-5fef-499f-a2cb-e38a0fee97f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:28.288525 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed3057-5fef-499f-a2cb-e38a0fee97f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:28.289609 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed3057-5fef-499f-a2cb-e38a0fee97f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:28.292305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed3057-5fef-499f-a2cb-e38a0fee97f3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:28.306592 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1051/2073] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:26:28.311035 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed3057-5fef-499f-a2cb-e38a0fee97f3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:28.337941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed3057-5fef-499f-a2cb-e38a0fee97f3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:28.349588 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dbb7ec-b272-42e8-8ae1-ac83b12bc9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:28.350417 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbb7ec-b272-42e8-8ae1-ac83b12bc9ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:28.351349 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbb7ec-b272-42e8-8ae1-ac83b12bc9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:28.352455 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbb7ec-b272-42e8-8ae1-ac83b12bc9ed admin admin] PATH_INFO: `/v3/projects/2c85a4108cff482ea9c503e0e2ffb0ab/users/1168299020094117ac5fef393f1878ee/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:28.355665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dbb7ec-b272-42e8-8ae1-ac83b12bc9ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c85a4108cff482ea9c503e0e2ffb0ab, user_id=1168299020094117ac5fef393f1878ee, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:28.359632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dbb7ec-b272-42e8-8ae1-ac83b12bc9ed admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:28.383886 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1052/2074] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:26:28 2022] PUT /identity/v3/projects/2c85a4108cff482ea9c503e0e2ffb0ab/users/1168299020094117ac5fef393f1878ee/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:26:28.401731 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009c08f-69d0-4dab-93df-52665dcb46b6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:28.405762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009c08f-69d0-4dab-93df-52665dcb46b6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:28.407091 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009c08f-69d0-4dab-93df-52665dcb46b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:28.449845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9843dc6-06b5-43e4-9b65-26f9cb17db4b None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:28.461770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-4009c08f-69d0-4dab-93df-52665dcb46b6 None None] MFA Rules not processed for user `1168299020094117ac5fef393f1878ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:28.463446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4009c08f-69d0-4dab-93df-52665dcb46b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:28.549001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b667676-fc08-4a2c-b3a0-88e076ad6a84 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2211e237fe554a82baf129e440a0fb43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:28.550327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b667676-fc08-4a2c-b3a0-88e076ad6a84 service glance] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:28.550622 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b667676-fc08-4a2c-b3a0-88e076ad6a84 service glance] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:28.550906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b667676-fc08-4a2c-b3a0-88e076ad6a84 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:28.553210 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0b667676-fc08-4a2c-b3a0-88e076ad6a84 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:28.581164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1053/2075] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:26:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:28.602307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009c08f-69d0-4dab-93df-52665dcb46b6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:28.604191 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4009c08f-69d0-4dab-93df-52665dcb46b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:28.653700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b667676-fc08-4a2c-b3a0-88e076ad6a84 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:28.655803 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b667676-fc08-4a2c-b3a0-88e076ad6a84 service glance] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:28.670340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1023/2076] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:26:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:26:28.694400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009c08f-69d0-4dab-93df-52665dcb46b6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:28.702071 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2d007-9ade-4d4e-8ed8-f5fbdc2de5e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:28.702619 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2d007-9ade-4d4e-8ed8-f5fbdc2de5e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:28.702908 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2d007-9ade-4d4e-8ed8-f5fbdc2de5e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:28.703198 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2d007-9ade-4d4e-8ed8-f5fbdc2de5e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:28.713427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2d007-9ade-4d4e-8ed8-f5fbdc2de5e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:28.715766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2d007-9ade-4d4e-8ed8-f5fbdc2de5e0 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:28.727402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1054/2077] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:26:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:26:28.836640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b667676-fc08-4a2c-b3a0-88e076ad6a84 service glance] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:28.852491 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b667676-fc08-4a2c-b3a0-88e076ad6a84 service glance] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:28.859754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bc1da-cabc-4cc9-8219-45f3a6bde636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:28.860589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bc1da-cabc-4cc9-8219-45f3a6bde636 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:28.861157 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bc1da-cabc-4cc9-8219-45f3a6bde636 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:28.862165 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bc1da-cabc-4cc9-8219-45f3a6bde636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:28.864053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6463a5-4581-49e4-8e9f-7067c63b0496 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:28.864760 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6463a5-4581-49e4-8e9f-7067c63b0496 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:28.864760 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6463a5-4581-49e4-8e9f-7067c63b0496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:28.887678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6a6463a5-4581-49e4-8e9f-7067c63b0496 None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:28.890037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6a6463a5-4581-49e4-8e9f-7067c63b0496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:28.923945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:28.951494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337bc1da-cabc-4cc9-8219-45f3a6bde636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:28.953047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337bc1da-cabc-4cc9-8219-45f3a6bde636 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:28.959426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:28.965853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1024/2078] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:26:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:26:28.973539 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1055/2079] 10.222.0.88 () {58 vars in 1009 bytes} [Tue Aug 23 22:26:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:29.044942 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bc1da-cabc-4cc9-8219-45f3a6bde636 service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:29.046439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-337bc1da-cabc-4cc9-8219-45f3a6bde636 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:29.189453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bc1da-cabc-4cc9-8219-45f3a6bde636 service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:29.198789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837e2a5-0186-460d-aa7e-131dbdc60bcd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:29.199687 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837e2a5-0186-460d-aa7e-131dbdc60bcd service swift] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:29.200629 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837e2a5-0186-460d-aa7e-131dbdc60bcd service swift] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:29.200629 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837e2a5-0186-460d-aa7e-131dbdc60bcd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:29.212557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837e2a5-0186-460d-aa7e-131dbdc60bcd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:29.215000 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837e2a5-0186-460d-aa7e-131dbdc60bcd service swift] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:29.237940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1025/2080] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:26:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 22:26:38.971717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:39.225617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=40 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:41.186022 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1336083-cc43-4db5-ae4c-840881e30872 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:41.186628 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1336083-cc43-4db5-ae4c-840881e30872 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:41.186628 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1336083-cc43-4db5-ae4c-840881e30872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:41.207783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a1336083-cc43-4db5-ae4c-840881e30872 None None] MFA Rules not processed for user `eb81364013af425e9956dd09ceb150e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:41.209205 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a1336083-cc43-4db5-ae4c-840881e30872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:41.217507 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1056/2081] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:26:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:48.802115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62dd1cc-1cc5-4d60-b361-9ab4b0838cd9 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:48.802657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62dd1cc-1cc5-4d60-b361-9ab4b0838cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:48.802923 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62dd1cc-1cc5-4d60-b361-9ab4b0838cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:48.896898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-d62dd1cc-1cc5-4d60-b361-9ab4b0838cd9 None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:48.899232 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d62dd1cc-1cc5-4d60-b361-9ab4b0838cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:48.916414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1026/2082] 10.222.0.88 () {58 vars in 1009 bytes} [Tue Aug 23 22:26:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:48.977773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1336083-cc43-4db5-ae4c-840881e30872 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:49.031381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1336083-cc43-4db5-ae4c-840881e30872 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:49.033590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62dd1cc-1cc5-4d60-b361-9ab4b0838cd9 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:49.038032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d62dd1cc-1cc5-4d60-b361-9ab4b0838cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:49.049616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f48f9-6385-4d48-9f4e-8ca3a2a09281 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:49.050672 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f48f9-6385-4d48-9f4e-8ca3a2a09281 service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:49.051162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f48f9-6385-4d48-9f4e-8ca3a2a09281 service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:49.055516 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f48f9-6385-4d48-9f4e-8ca3a2a09281 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:49.091565 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f48f9-6385-4d48-9f4e-8ca3a2a09281 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:49.098392 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f48f9-6385-4d48-9f4e-8ca3a2a09281 service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:49.139468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1057/2083] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:26:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:26:49.274241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62dd1cc-1cc5-4d60-b361-9ab4b0838cd9 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:49.281145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3bfa2-e18a-46ba-a4e2-689b9130ea06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:49.281872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3bfa2-e18a-46ba-a4e2-689b9130ea06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:49.282156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3bfa2-e18a-46ba-a4e2-689b9130ea06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:49.282440 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3bfa2-e18a-46ba-a4e2-689b9130ea06 admin admin] PATH_INFO: `/v3/users/d278d1d334f2424ea82f97229e1224b0` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:49.283996 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3bfa2-e18a-46ba-a4e2-689b9130ea06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d278d1d334f2424ea82f97229e1224b0)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:49.287009 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3bfa2-e18a-46ba-a4e2-689b9130ea06 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:49.544467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e1d3bfa2-e18a-46ba-a4e2-689b9130ea06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd278d1d334f2424ea82f97229e1224b0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:49.547682 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e1d3bfa2-e18a-46ba-a4e2-689b9130ea06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd278d1d334f2424ea82f97229e1224b0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:49.596281 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e1d3bfa2-e18a-46ba-a4e2-689b9130ea06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd278d1d334f2424ea82f97229e1224b0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:49.668052 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e1d3bfa2-e18a-46ba-a4e2-689b9130ea06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd278d1d334f2424ea82f97229e1224b0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:49.721644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e1d3bfa2-e18a-46ba-a4e2-689b9130ea06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd278d1d334f2424ea82f97229e1224b0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:49.766255 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1027/2084] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:26:49 2022] DELETE /identity/v3/users/d278d1d334f2424ea82f97229e1224b0 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:26:49.772029 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f125a-6f3f-4b34-bbf5-d5880ba2b870 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:49.772562 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f125a-6f3f-4b34-bbf5-d5880ba2b870 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:49.773025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f125a-6f3f-4b34-bbf5-d5880ba2b870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:49.839368 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-be2f125a-6f3f-4b34-bbf5-d5880ba2b870 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:49.847370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-be2f125a-6f3f-4b34-bbf5-d5880ba2b870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:50.274390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1058/2085] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 504 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:50.288971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3bfa2-e18a-46ba-a4e2-689b9130ea06 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:50.292406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e1d3bfa2-e18a-46ba-a4e2-689b9130ea06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:50.599607 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3bfa2-e18a-46ba-a4e2-689b9130ea06 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:50.606196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f503443-7ae5-4fcb-8497-b15133897b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:50.607037 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f503443-7ae5-4fcb-8497-b15133897b73 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:50.607428 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f503443-7ae5-4fcb-8497-b15133897b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:50.607933 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f503443-7ae5-4fcb-8497-b15133897b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:50.654338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f503443-7ae5-4fcb-8497-b15133897b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:50.657300 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f503443-7ae5-4fcb-8497-b15133897b73 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:50.701339 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1028/2086] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:26:52.948992 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f125a-6f3f-4b34-bbf5-d5880ba2b870 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:52.951445 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-be2f125a-6f3f-4b34-bbf5-d5880ba2b870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:52.981457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f125a-6f3f-4b34-bbf5-d5880ba2b870 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:52.991910 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d149d2-a84c-414d-b803-10b7f5913aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:52.995804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d149d2-a84c-414d-b803-10b7f5913aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:52.996382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d149d2-a84c-414d-b803-10b7f5913aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:52.996955 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d149d2-a84c-414d-b803-10b7f5913aa5 admin admin] PATH_INFO: `/v3/projects/40b955b963bd44199d476f1451842bde` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:52.999621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d149d2-a84c-414d-b803-10b7f5913aa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40b955b963bd44199d476f1451842bde)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:53.009450 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d149d2-a84c-414d-b803-10b7f5913aa5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:53.102107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-29d149d2-a84c-414d-b803-10b7f5913aa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40b955b963bd44199d476f1451842bde', 'request_id': 'req-29d149d2-a84c-414d-b803-10b7f5913aa5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:53.113943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-29d149d2-a84c-414d-b803-10b7f5913aa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40b955b963bd44199d476f1451842bde', 'request_id': 'req-29d149d2-a84c-414d-b803-10b7f5913aa5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:53.126945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-29d149d2-a84c-414d-b803-10b7f5913aa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40b955b963bd44199d476f1451842bde', 'request_id': 'req-29d149d2-a84c-414d-b803-10b7f5913aa5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:53.143155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-29d149d2-a84c-414d-b803-10b7f5913aa5 admin admin] The token cache is being invalidate because project 40b955b963bd44199d476f1451842bde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:26:53.143471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-29d149d2-a84c-414d-b803-10b7f5913aa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:53.144189 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-29d149d2-a84c-414d-b803-10b7f5913aa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:26:53.146124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1059/2087] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:26:52 2022] DELETE /identity/v3/projects/40b955b963bd44199d476f1451842bde => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:26:53.153777 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448eb634-c62a-47aa-ad68-bdedad7538dd None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:53.154330 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448eb634-c62a-47aa-ad68-bdedad7538dd None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:53.154793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448eb634-c62a-47aa-ad68-bdedad7538dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:53.189561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-448eb634-c62a-47aa-ad68-bdedad7538dd None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:53.191047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-448eb634-c62a-47aa-ad68-bdedad7538dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:53.203580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:53.280614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1029/2088] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:53.292575 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d4ed2-c642-4ef9-bf29-ac137f7b469c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:53.292999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d4ed2-c642-4ef9-bf29-ac137f7b469c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:53.293380 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d4ed2-c642-4ef9-bf29-ac137f7b469c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:53.316282 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2c6d4ed2-c642-4ef9-bf29-ac137f7b469c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:53.317446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2c6d4ed2-c642-4ef9-bf29-ac137f7b469c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:53.327619 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1060/2089] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:53.332704 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448eb634-c62a-47aa-ad68-bdedad7538dd None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:53.347312 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448eb634-c62a-47aa-ad68-bdedad7538dd None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:53.352880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f5fb5-7a6a-41b9-8fb1-929799d4416c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:53.353658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f5fb5-7a6a-41b9-8fb1-929799d4416c admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:53.353951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f5fb5-7a6a-41b9-8fb1-929799d4416c admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:53.354247 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f5fb5-7a6a-41b9-8fb1-929799d4416c admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:53.354929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f5fb5-7a6a-41b9-8fb1-929799d4416c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:26:53.355388 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f5fb5-7a6a-41b9-8fb1-929799d4416c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:53.356862 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f5fb5-7a6a-41b9-8fb1-929799d4416c admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:53.377583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1030/2090] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:26:53.379199 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d4ed2-c642-4ef9-bf29-ac137f7b469c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:53.396369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d4ed2-c642-4ef9-bf29-ac137f7b469c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:53.405150 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac97721-6d55-44b4-9ebc-30eec1eb0626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:53.406295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac97721-6d55-44b4-9ebc-30eec1eb0626 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:53.406804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac97721-6d55-44b4-9ebc-30eec1eb0626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:53.407468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac97721-6d55-44b4-9ebc-30eec1eb0626 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:53.408938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac97721-6d55-44b4-9ebc-30eec1eb0626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:53.411657 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac97721-6d55-44b4-9ebc-30eec1eb0626 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:53.471649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1061/2091] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:26:53 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:26:53.477486 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f5fb5-7a6a-41b9-8fb1-929799d4416c admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:53.495945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f5fb5-7a6a-41b9-8fb1-929799d4416c admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:53.602322 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79e912-a1ff-4ab2-bf0e-1ad028fd6b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:53.602984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79e912-a1ff-4ab2-bf0e-1ad028fd6b3b admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:53.604002 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79e912-a1ff-4ab2-bf0e-1ad028fd6b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:53.604289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79e912-a1ff-4ab2-bf0e-1ad028fd6b3b admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:53.604988 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79e912-a1ff-4ab2-bf0e-1ad028fd6b3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:53.607098 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79e912-a1ff-4ab2-bf0e-1ad028fd6b3b admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:53.640656 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1031/2092] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:26:53 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:26:53.646049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac97721-6d55-44b4-9ebc-30eec1eb0626 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:53.658049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac97721-6d55-44b4-9ebc-30eec1eb0626 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:53.665313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76c9e1-4ac4-42c3-a85d-a9a79a05fa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:53.666657 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76c9e1-4ac4-42c3-a85d-a9a79a05fa90 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:53.666946 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76c9e1-4ac4-42c3-a85d-a9a79a05fa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:53.667755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76c9e1-4ac4-42c3-a85d-a9a79a05fa90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:53.668982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76c9e1-4ac4-42c3-a85d-a9a79a05fa90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:53.671347 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76c9e1-4ac4-42c3-a85d-a9a79a05fa90 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:53.688505 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1062/2093] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:26:53.693088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79e912-a1ff-4ab2-bf0e-1ad028fd6b3b admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:53.705299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79e912-a1ff-4ab2-bf0e-1ad028fd6b3b admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:53.710834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07dc37-1e81-491a-b7d6-4df292e3face admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:53.711468 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07dc37-1e81-491a-b7d6-4df292e3face admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:53.711752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07dc37-1e81-491a-b7d6-4df292e3face admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:53.712022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07dc37-1e81-491a-b7d6-4df292e3face admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:53.713352 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07dc37-1e81-491a-b7d6-4df292e3face admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:53.715140 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07dc37-1e81-491a-b7d6-4df292e3face admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:53.727665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1032/2094] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:26:53.732914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76c9e1-4ac4-42c3-a85d-a9a79a05fa90 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:53.747811 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76c9e1-4ac4-42c3-a85d-a9a79a05fa90 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:53.754816 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e97286-6357-4cce-a045-253385312548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:53.756084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e97286-6357-4cce-a045-253385312548 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:53.756657 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e97286-6357-4cce-a045-253385312548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:53.757102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e97286-6357-4cce-a045-253385312548 admin admin] PATH_INFO: `/v3/projects/ece858375eb0450e80f794c0c5bcc7da/users/6df7d6b5ae6541ebb613516403495a36/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:53.786175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e97286-6357-4cce-a045-253385312548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ece858375eb0450e80f794c0c5bcc7da, user_id=6df7d6b5ae6541ebb613516403495a36, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:53.789119 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e97286-6357-4cce-a045-253385312548 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:53.812961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:53.837247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1063/2095] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:26:53 2022] PUT /identity/v3/projects/ece858375eb0450e80f794c0c5bcc7da/users/6df7d6b5ae6541ebb613516403495a36/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:26:53.844936 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c5acf-2044-4713-8083-dc0870463dce None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:53.845862 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c5acf-2044-4713-8083-dc0870463dce None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:53.846356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c5acf-2044-4713-8083-dc0870463dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:53.895204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-dd5c5acf-2044-4713-8083-dc0870463dce None None] MFA Rules not processed for user `6df7d6b5ae6541ebb613516403495a36`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:53.896862 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-dd5c5acf-2044-4713-8083-dc0870463dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:54.006707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1033/2096] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:26:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:26:54.019647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e97286-6357-4cce-a045-253385312548 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:54.021455 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-72e97286-6357-4cce-a045-253385312548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:54.098095 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e97286-6357-4cce-a045-253385312548 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:54.101734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ae393-bfa6-4fec-bd84-5f54760a4c6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:54.102350 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ae393-bfa6-4fec-bd84-5f54760a4c6e service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:54.102629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ae393-bfa6-4fec-bd84-5f54760a4c6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:54.102905 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ae393-bfa6-4fec-bd84-5f54760a4c6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:54.113470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ae393-bfa6-4fec-bd84-5f54760a4c6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:54.114977 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ae393-bfa6-4fec-bd84-5f54760a4c6e service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:54.125254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1064/2097] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:26:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:26:57.094051 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0e81a-9b8b-4036-bf8c-25cc0096c80f None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:57.095368 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0e81a-9b8b-4036-bf8c-25cc0096c80f None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:57.096190 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0e81a-9b8b-4036-bf8c-25cc0096c80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:57.155969 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-aeb0e81a-9b8b-4036-bf8c-25cc0096c80f None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:26:57.158344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-aeb0e81a-9b8b-4036-bf8c-25cc0096c80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:57.170625 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:57.405661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1034/2098] 10.222.0.88 () {58 vars in 1009 bytes} [Tue Aug 23 22:26:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:26:57.421934 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ae393-bfa6-4fec-bd84-5f54760a4c6e service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:26:57.424643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ab2ae393-bfa6-4fec-bd84-5f54760a4c6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:26:57.690444 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ae393-bfa6-4fec-bd84-5f54760a4c6e service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:26:57.707102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806f89d-db20-4c17-b613-5eb774d946fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:26:57.708772 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806f89d-db20-4c17-b613-5eb774d946fa service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:26:57.709568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806f89d-db20-4c17-b613-5eb774d946fa service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:26:57.710313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806f89d-db20-4c17-b613-5eb774d946fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:26:57.731094 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806f89d-db20-4c17-b613-5eb774d946fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:26:57.736283 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806f89d-db20-4c17-b613-5eb774d946fa service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:26:57.778938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1065/2099] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:26:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:27:07.387904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=20 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:07.765211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=58 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:13.365152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0e81a-9b8b-4036-bf8c-25cc0096c80f None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:13.518380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0e81a-9b8b-4036-bf8c-25cc0096c80f None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:13.548008 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db9cb6-7541-4d63-8f9c-7f4cb5ec4a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:13.560263 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db9cb6-7541-4d63-8f9c-7f4cb5ec4a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:13.561091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db9cb6-7541-4d63-8f9c-7f4cb5ec4a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:13.561465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db9cb6-7541-4d63-8f9c-7f4cb5ec4a08 admin admin] PATH_INFO: `/v3/users/6df7d6b5ae6541ebb613516403495a36` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:13.566317 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db9cb6-7541-4d63-8f9c-7f4cb5ec4a08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6df7d6b5ae6541ebb613516403495a36)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:13.568544 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db9cb6-7541-4d63-8f9c-7f4cb5ec4a08 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:13.694968 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-24db9cb6-7541-4d63-8f9c-7f4cb5ec4a08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6df7d6b5ae6541ebb613516403495a36'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:13.696015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-24db9cb6-7541-4d63-8f9c-7f4cb5ec4a08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6df7d6b5ae6541ebb613516403495a36'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:13.718325 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-24db9cb6-7541-4d63-8f9c-7f4cb5ec4a08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6df7d6b5ae6541ebb613516403495a36'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:13.753955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-24db9cb6-7541-4d63-8f9c-7f4cb5ec4a08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6df7d6b5ae6541ebb613516403495a36'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:13.772471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-24db9cb6-7541-4d63-8f9c-7f4cb5ec4a08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6df7d6b5ae6541ebb613516403495a36'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:13.790689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1035/2100] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:27:13 2022] DELETE /identity/v3/users/6df7d6b5ae6541ebb613516403495a36 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:13.799799 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f42c3f-2a8a-446a-9b77-309cf176a6bd None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:13.800290 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f42c3f-2a8a-446a-9b77-309cf176a6bd None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:13.800789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f42c3f-2a8a-446a-9b77-309cf176a6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:13.845838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-79f42c3f-2a8a-446a-9b77-309cf176a6bd None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:13.848305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-79f42c3f-2a8a-446a-9b77-309cf176a6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:14.017273 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1066/2101] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:27:14.029796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db9cb6-7541-4d63-8f9c-7f4cb5ec4a08 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:14.031323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-24db9cb6-7541-4d63-8f9c-7f4cb5ec4a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:14.132692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db9cb6-7541-4d63-8f9c-7f4cb5ec4a08 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:14.138872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811d561-3948-41ef-a98a-3e2b100f946b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:14.141023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811d561-3948-41ef-a98a-3e2b100f946b service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:14.141531 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811d561-3948-41ef-a98a-3e2b100f946b service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:14.141989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811d561-3948-41ef-a98a-3e2b100f946b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:14.153650 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811d561-3948-41ef-a98a-3e2b100f946b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:14.155374 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811d561-3948-41ef-a98a-3e2b100f946b service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:14.168506 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1036/2102] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:27:14.302210 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b311464-a47a-4cf7-af1e-234510dc1a3e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:14.303560 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b311464-a47a-4cf7-af1e-234510dc1a3e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:14.304828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b311464-a47a-4cf7-af1e-234510dc1a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:14.505792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6b311464-a47a-4cf7-af1e-234510dc1a3e None None] MFA Rules not processed for user `1db1a9a77be6452991ec4f99af814e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:14.520754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6b311464-a47a-4cf7-af1e-234510dc1a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:14.832353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1067/2103] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 533 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:14.837549 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993193f-d344-4207-85d7-46631e7439ad None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:14.838110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993193f-d344-4207-85d7-46631e7439ad None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:14.838273 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993193f-d344-4207-85d7-46631e7439ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:14.913825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-c993193f-d344-4207-85d7-46631e7439ad None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:14.915549 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c993193f-d344-4207-85d7-46631e7439ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:14.973778 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:15.088851 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1037/2104] 10.222.0.88 () {58 vars in 1009 bytes} [Tue Aug 23 22:27:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:15.374318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b311464-a47a-4cf7-af1e-234510dc1a3e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:15.376259 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6b311464-a47a-4cf7-af1e-234510dc1a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:15.504409 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993193f-d344-4207-85d7-46631e7439ad None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:15.506216 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c993193f-d344-4207-85d7-46631e7439ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:15.530503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993193f-d344-4207-85d7-46631e7439ad None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:15.541556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c1297-e5ee-41a5-a0b2-dc3e9b30ee38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:15.542614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c1297-e5ee-41a5-a0b2-dc3e9b30ee38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:15.542907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c1297-e5ee-41a5-a0b2-dc3e9b30ee38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:15.543915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c1297-e5ee-41a5-a0b2-dc3e9b30ee38 admin admin] PATH_INFO: `/v3/projects/ece858375eb0450e80f794c0c5bcc7da` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:15.544872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c1297-e5ee-41a5-a0b2-dc3e9b30ee38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ece858375eb0450e80f794c0c5bcc7da)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:15.547178 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c1297-e5ee-41a5-a0b2-dc3e9b30ee38 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:15.629514 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b311464-a47a-4cf7-af1e-234510dc1a3e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:15.643287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8d9c1297-e5ee-41a5-a0b2-dc3e9b30ee38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ece858375eb0450e80f794c0c5bcc7da', 'request_id': 'req-8d9c1297-e5ee-41a5-a0b2-dc3e9b30ee38'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:15.653935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8d9c1297-e5ee-41a5-a0b2-dc3e9b30ee38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ece858375eb0450e80f794c0c5bcc7da', 'request_id': 'req-8d9c1297-e5ee-41a5-a0b2-dc3e9b30ee38'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:15.664377 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8d9c1297-e5ee-41a5-a0b2-dc3e9b30ee38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ece858375eb0450e80f794c0c5bcc7da', 'request_id': 'req-8d9c1297-e5ee-41a5-a0b2-dc3e9b30ee38'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:15.695751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8d9c1297-e5ee-41a5-a0b2-dc3e9b30ee38 admin admin] The token cache is being invalidate because project ece858375eb0450e80f794c0c5bcc7da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:27:15.696047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8d9c1297-e5ee-41a5-a0b2-dc3e9b30ee38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:15.698862 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8d9c1297-e5ee-41a5-a0b2-dc3e9b30ee38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:15.702277 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1038/2105] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:27:15 2022] DELETE /identity/v3/projects/ece858375eb0450e80f794c0c5bcc7da => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:15.761019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:15.804795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c7872-8e90-41d0-a935-04d22684abd3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:15.806400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c7872-8e90-41d0-a935-04d22684abd3 service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:15.806963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c7872-8e90-41d0-a935-04d22684abd3 service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:15.808388 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c7872-8e90-41d0-a935-04d22684abd3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:15.829413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8c7c7872-8e90-41d0-a935-04d22684abd3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:16.019674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c7872-8e90-41d0-a935-04d22684abd3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:16.022824 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c7872-8e90-41d0-a935-04d22684abd3 service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:16.041038 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1068/2106] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:27:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 667 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 22:27:21.775548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef161c-533b-40b5-a6b2-5f7d573d7dc5 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:21.776260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef161c-533b-40b5-a6b2-5f7d573d7dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:21.776260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef161c-533b-40b5-a6b2-5f7d573d7dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:21.822893 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-a5ef161c-533b-40b5-a6b2-5f7d573d7dc5 None None] MFA Rules not processed for user `78b05a538ce0466ea8b3e27a0a6fc162`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:21.824425 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a5ef161c-533b-40b5-a6b2-5f7d573d7dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:22.004793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1039/2107] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:25.931650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c7872-8e90-41d0-a935-04d22684abd3 service swift] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:25.935468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8c7c7872-8e90-41d0-a935-04d22684abd3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:26.096571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c7872-8e90-41d0-a935-04d22684abd3 service swift] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:26.103426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b22f5-5c85-42f6-904b-44cf610895cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:26.104164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b22f5-5c85-42f6-904b-44cf610895cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:26.104618 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b22f5-5c85-42f6-904b-44cf610895cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:26.105033 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b22f5-5c85-42f6-904b-44cf610895cf admin admin] PATH_INFO: `/v3/users/0b5d58cdcf284732a838a7b04e9c2a51` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:26.106904 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b22f5-5c85-42f6-904b-44cf610895cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b5d58cdcf284732a838a7b04e9c2a51)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:26.113178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b22f5-5c85-42f6-904b-44cf610895cf admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:26.233490 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d08b22f5-5c85-42f6-904b-44cf610895cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b5d58cdcf284732a838a7b04e9c2a51'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:26.257950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d08b22f5-5c85-42f6-904b-44cf610895cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b5d58cdcf284732a838a7b04e9c2a51'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:26.272785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d08b22f5-5c85-42f6-904b-44cf610895cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b5d58cdcf284732a838a7b04e9c2a51'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:26.275275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d08b22f5-5c85-42f6-904b-44cf610895cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b5d58cdcf284732a838a7b04e9c2a51'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:26.292791 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d08b22f5-5c85-42f6-904b-44cf610895cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b5d58cdcf284732a838a7b04e9c2a51'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:26.308660 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1069/2108] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:27:25 2022] DELETE /identity/v3/users/0b5d58cdcf284732a838a7b04e9c2a51 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:26.310994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980642b5-4317-4beb-a099-759543fa28b9 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:26.311575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980642b5-4317-4beb-a099-759543fa28b9 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:26.311925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980642b5-4317-4beb-a099-759543fa28b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:26.347992 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-980642b5-4317-4beb-a099-759543fa28b9 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:26.353932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-980642b5-4317-4beb-a099-759543fa28b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:26.472252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:26.537117 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1040/2109] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:26.551840 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b22f5-5c85-42f6-904b-44cf610895cf admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:26.552618 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d08b22f5-5c85-42f6-904b-44cf610895cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:26.577436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:26.770031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b22f5-5c85-42f6-904b-44cf610895cf admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:26.782040 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802f2a6-b009-4e41-972c-100ee46a9221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:26.783249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802f2a6-b009-4e41-972c-100ee46a9221 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:26.783707 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802f2a6-b009-4e41-972c-100ee46a9221 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:26.785530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802f2a6-b009-4e41-972c-100ee46a9221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:26.810666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802f2a6-b009-4e41-972c-100ee46a9221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:26.815774 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802f2a6-b009-4e41-972c-100ee46a9221 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:26.847358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1070/2110] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:27:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:27:27.445254 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980642b5-4317-4beb-a099-759543fa28b9 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:27.449449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-980642b5-4317-4beb-a099-759543fa28b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:27.481728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980642b5-4317-4beb-a099-759543fa28b9 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:27.501165 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:27.503340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:27.503890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:27.504507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e admin admin] PATH_INFO: `/v3/projects/2dfca0522378453093b94d29d617fc73` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:27.506776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dfca0522378453093b94d29d617fc73)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:27.511361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:27.620601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dfca0522378453093b94d29d617fc73', 'request_id': 'req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:27.639098 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dfca0522378453093b94d29d617fc73', 'request_id': 'req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:27.656023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dfca0522378453093b94d29d617fc73', 'request_id': 'req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:27.692460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e admin admin] The token cache is being invalidate because project 2dfca0522378453093b94d29d617fc73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:27:27.693434 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:27.694480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:27.701283 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1041/2111] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:27:27 2022] DELETE /identity/v3/projects/2dfca0522378453093b94d29d617fc73 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:27.703166 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802f2a6-b009-4e41-972c-100ee46a9221 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:27.729674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802f2a6-b009-4e41-972c-100ee46a9221 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:27.872665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335dc52f-df4b-445c-99ee-5040d15715d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:27.874247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335dc52f-df4b-445c-99ee-5040d15715d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:27.874608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335dc52f-df4b-445c-99ee-5040d15715d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:27.874957 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335dc52f-df4b-445c-99ee-5040d15715d9 admin admin] PATH_INFO: `/v3/users/1168299020094117ac5fef393f1878ee` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:27.876926 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335dc52f-df4b-445c-99ee-5040d15715d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1168299020094117ac5fef393f1878ee)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:27.879997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335dc52f-df4b-445c-99ee-5040d15715d9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:28.034124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-335dc52f-df4b-445c-99ee-5040d15715d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1168299020094117ac5fef393f1878ee'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:28.074846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-335dc52f-df4b-445c-99ee-5040d15715d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1168299020094117ac5fef393f1878ee'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:28.096084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-335dc52f-df4b-445c-99ee-5040d15715d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1168299020094117ac5fef393f1878ee'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:28.096729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-335dc52f-df4b-445c-99ee-5040d15715d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1168299020094117ac5fef393f1878ee'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:28.107781 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-335dc52f-df4b-445c-99ee-5040d15715d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1168299020094117ac5fef393f1878ee'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:28.122404 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1071/2112] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:27:27 2022] DELETE /identity/v3/users/1168299020094117ac5fef393f1878ee => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:28.678916 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:28.681459 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:28.708144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceee099-5dc8-4b16-8377-e9f6e2118c7e admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:28.882786 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6459fd0-663e-4a38-b1d2-10369c8ced98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:28.884860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6459fd0-663e-4a38-b1d2-10369c8ced98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:28.886340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6459fd0-663e-4a38-b1d2-10369c8ced98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:28.886967 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6459fd0-663e-4a38-b1d2-10369c8ced98 admin admin] PATH_INFO: `/v3/projects/2c85a4108cff482ea9c503e0e2ffb0ab` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:28.889309 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6459fd0-663e-4a38-b1d2-10369c8ced98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c85a4108cff482ea9c503e0e2ffb0ab)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:28.892399 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6459fd0-663e-4a38-b1d2-10369c8ced98 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:29.018398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d6459fd0-663e-4a38-b1d2-10369c8ced98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c85a4108cff482ea9c503e0e2ffb0ab', 'request_id': 'req-d6459fd0-663e-4a38-b1d2-10369c8ced98'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:29.047605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d6459fd0-663e-4a38-b1d2-10369c8ced98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c85a4108cff482ea9c503e0e2ffb0ab', 'request_id': 'req-d6459fd0-663e-4a38-b1d2-10369c8ced98'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:29.073556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d6459fd0-663e-4a38-b1d2-10369c8ced98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c85a4108cff482ea9c503e0e2ffb0ab', 'request_id': 'req-d6459fd0-663e-4a38-b1d2-10369c8ced98'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:29.115333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d6459fd0-663e-4a38-b1d2-10369c8ced98 admin admin] The token cache is being invalidate because project 2c85a4108cff482ea9c503e0e2ffb0ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:27:29.115922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d6459fd0-663e-4a38-b1d2-10369c8ced98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:29.117223 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d6459fd0-663e-4a38-b1d2-10369c8ced98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:29.121494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1042/2113] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:27:28 2022] DELETE /identity/v3/projects/2c85a4108cff482ea9c503e0e2ffb0ab => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:29.137382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8e118-a7ff-4fee-b80f-5222a4e92f72 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:29.138236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8e118-a7ff-4fee-b80f-5222a4e92f72 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:29.138387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8e118-a7ff-4fee-b80f-5222a4e92f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:29.197258 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d5d8e118-a7ff-4fee-b80f-5222a4e92f72 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:29.199532 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d5d8e118-a7ff-4fee-b80f-5222a4e92f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:29.286495 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:29.414163 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1072/2114] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:29.417672 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42e9d3-723a-4664-beb0-122476fb0dee None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:29.418270 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42e9d3-723a-4664-beb0-122476fb0dee None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:29.418842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42e9d3-723a-4664-beb0-122476fb0dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:29.485604 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-7c42e9d3-723a-4664-beb0-122476fb0dee None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:29.488208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-7c42e9d3-723a-4664-beb0-122476fb0dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:29.521967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8e118-a7ff-4fee-b80f-5222a4e92f72 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:29.532363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1043/2115] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:29.546265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8e118-a7ff-4fee-b80f-5222a4e92f72 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:29.558272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8ad39-9381-481e-994a-6a773a85c482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:29.560559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8ad39-9381-481e-994a-6a773a85c482 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:29.562846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8ad39-9381-481e-994a-6a773a85c482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:29.563561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8ad39-9381-481e-994a-6a773a85c482 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:29.567960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8ad39-9381-481e-994a-6a773a85c482 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:27:29.568707 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8ad39-9381-481e-994a-6a773a85c482 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:29.579464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8ad39-9381-481e-994a-6a773a85c482 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:29.625908 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1073/2116] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:27:29.630579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42e9d3-723a-4664-beb0-122476fb0dee None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:29.644149 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:29.682087 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42e9d3-723a-4664-beb0-122476fb0dee None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:29.714872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d367b14-1950-4386-ae7b-04c31b03a4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:29.715990 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d367b14-1950-4386-ae7b-04c31b03a4b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:29.716471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d367b14-1950-4386-ae7b-04c31b03a4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:29.717186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d367b14-1950-4386-ae7b-04c31b03a4b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:29.718964 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d367b14-1950-4386-ae7b-04c31b03a4b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:29.736216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f93e8e-4fd7-4e1f-8ba8-3ad8cecc982f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:29.736863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f93e8e-4fd7-4e1f-8ba8-3ad8cecc982f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:29.737191 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d367b14-1950-4386-ae7b-04c31b03a4b4 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:29.737628 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f93e8e-4fd7-4e1f-8ba8-3ad8cecc982f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:29.831937 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-60f93e8e-4fd7-4e1f-8ba8-3ad8cecc982f None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:29.835323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1044/2117] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:27:29 2022] POST /identity/v3/projects => generated 379 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:27:29.837854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-60f93e8e-4fd7-4e1f-8ba8-3ad8cecc982f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:29.842712 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d367b14-1950-4386-ae7b-04c31b03a4b4 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:30.068139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1074/2118] 10.222.0.88 () {58 vars in 967 bytes} [Tue Aug 23 22:27:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:30.130838 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d367b14-1950-4386-ae7b-04c31b03a4b4 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:30.149634 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5736398-6e0b-4910-aacd-27e5729c258e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:30.150646 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5736398-6e0b-4910-aacd-27e5729c258e admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:30.151494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5736398-6e0b-4910-aacd-27e5729c258e admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:30.152015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5736398-6e0b-4910-aacd-27e5729c258e admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:30.153012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5736398-6e0b-4910-aacd-27e5729c258e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:30.157148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5736398-6e0b-4910-aacd-27e5729c258e admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:30.220187 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f93e8e-4fd7-4e1f-8ba8-3ad8cecc982f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:30.223152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-60f93e8e-4fd7-4e1f-8ba8-3ad8cecc982f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:30.239252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1045/2119] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:27:29 2022] POST /identity/v3/users => generated 290 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:27:30.243900 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5736398-6e0b-4910-aacd-27e5729c258e admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:30.326437 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5736398-6e0b-4910-aacd-27e5729c258e admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:30.380681 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457419a-4557-4d46-ad82-3d145f2a2e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:30.381835 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457419a-4557-4d46-ad82-3d145f2a2e50 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:30.382354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457419a-4557-4d46-ad82-3d145f2a2e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:30.382854 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457419a-4557-4d46-ad82-3d145f2a2e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:30.384668 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457419a-4557-4d46-ad82-3d145f2a2e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:30.387972 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457419a-4557-4d46-ad82-3d145f2a2e50 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:30.484089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1046/2120] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:27:30.488485 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457419a-4557-4d46-ad82-3d145f2a2e50 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:30.557918 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457419a-4557-4d46-ad82-3d145f2a2e50 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:30.588554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca07ff-fa40-4a9d-8ea6-b96a55c745d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:30.589764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca07ff-fa40-4a9d-8ea6-b96a55c745d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:30.590298 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca07ff-fa40-4a9d-8ea6-b96a55c745d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:30.590823 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca07ff-fa40-4a9d-8ea6-b96a55c745d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:30.592082 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca07ff-fa40-4a9d-8ea6-b96a55c745d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:30.595289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca07ff-fa40-4a9d-8ea6-b96a55c745d2 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:30.661702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1047/2121] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:27:30.676832 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca07ff-fa40-4a9d-8ea6-b96a55c745d2 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:30.682495 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f93e8e-4fd7-4e1f-8ba8-3ad8cecc982f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:30.714412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b25be-7502-4971-888e-50aef3db9735 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:30.715454 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b25be-7502-4971-888e-50aef3db9735 service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:30.715954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b25be-7502-4971-888e-50aef3db9735 service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:30.716436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b25be-7502-4971-888e-50aef3db9735 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:30.729661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca07ff-fa40-4a9d-8ea6-b96a55c745d2 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:30.747598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726ad5c-8cfc-41ad-92d9-02c1d9966626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:30.748386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726ad5c-8cfc-41ad-92d9-02c1d9966626 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:30.753228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726ad5c-8cfc-41ad-92d9-02c1d9966626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:30.753513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726ad5c-8cfc-41ad-92d9-02c1d9966626 admin admin] PATH_INFO: `/v3/projects/a54a028703e64d69b5f726dc027703d8/users/deecf18eaec543988ab89a960ca7885b/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:30.781386 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b25be-7502-4971-888e-50aef3db9735 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:30.784522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b25be-7502-4971-888e-50aef3db9735 service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:30.815673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726ad5c-8cfc-41ad-92d9-02c1d9966626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a54a028703e64d69b5f726dc027703d8, user_id=deecf18eaec543988ab89a960ca7885b, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:30.819033 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726ad5c-8cfc-41ad-92d9-02c1d9966626 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:30.846512 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1075/2122] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:27:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 22:27:30.864721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1048/2123] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:27:30 2022] PUT /identity/v3/projects/a54a028703e64d69b5f726dc027703d8/users/deecf18eaec543988ab89a960ca7885b/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:30.892468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd91a38-0525-43f9-aada-11e4e0bffaba None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:30.893816 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd91a38-0525-43f9-aada-11e4e0bffaba None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:30.894355 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd91a38-0525-43f9-aada-11e4e0bffaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:31.036665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ffd91a38-0525-43f9-aada-11e4e0bffaba None None] MFA Rules not processed for user `deecf18eaec543988ab89a960ca7885b`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:31.046777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ffd91a38-0525-43f9-aada-11e4e0bffaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:31.234614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726ad5c-8cfc-41ad-92d9-02c1d9966626 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:31.236674 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3726ad5c-8cfc-41ad-92d9-02c1d9966626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:31.261523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:31.409747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1076/2124] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 520 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:31.417428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd91a38-0525-43f9-aada-11e4e0bffaba None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:31.560321 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726ad5c-8cfc-41ad-92d9-02c1d9966626 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:31.581170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b16b4-3061-4e36-b5c2-fe8a20230b2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e642dd7a07ca4afaa40c2769299d8bad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:31.582608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b16b4-3061-4e36-b5c2-fe8a20230b2f service placement] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:31.583109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b16b4-3061-4e36-b5c2-fe8a20230b2f service placement] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:31.583603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b16b4-3061-4e36-b5c2-fe8a20230b2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:31.590831 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-249b16b4-3061-4e36-b5c2-fe8a20230b2f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:31.658326 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b16b4-3061-4e36-b5c2-fe8a20230b2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:31.661675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b16b4-3061-4e36-b5c2-fe8a20230b2f service placement] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:31.699299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1049/2125] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:27:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:27:31.747202 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd91a38-0525-43f9-aada-11e4e0bffaba None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:31.773200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ab74e-9add-40af-b6b4-d4f8ba35d6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:31.774359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ab74e-9add-40af-b6b4-d4f8ba35d6aa admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:31.774939 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ab74e-9add-40af-b6b4-d4f8ba35d6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:31.775534 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ab74e-9add-40af-b6b4-d4f8ba35d6aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:31.776764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ab74e-9add-40af-b6b4-d4f8ba35d6aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:31.780722 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ab74e-9add-40af-b6b4-d4f8ba35d6aa admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:31.857289 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b16b4-3061-4e36-b5c2-fe8a20230b2f service placement] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:31.872252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1077/2126] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:27:31 2022] POST /identity/v3/projects => generated 379 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:27:31.885291 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b16b4-3061-4e36-b5c2-fe8a20230b2f service placement] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:31.911329 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:32.054858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63731792-9e7d-4cdc-adf9-b78305d170df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:32.056648 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63731792-9e7d-4cdc-adf9-b78305d170df admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:32.058339 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63731792-9e7d-4cdc-adf9-b78305d170df admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:32.059374 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63731792-9e7d-4cdc-adf9-b78305d170df admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:32.061025 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63731792-9e7d-4cdc-adf9-b78305d170df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:32.067432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63731792-9e7d-4cdc-adf9-b78305d170df admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:32.127683 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ab74e-9add-40af-b6b4-d4f8ba35d6aa admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:32.129647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1050/2127] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:27:31 2022] POST /identity/v3/users => generated 296 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:27:32.142153 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ab74e-9add-40af-b6b4-d4f8ba35d6aa admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:32.153558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb58399-8833-4a04-a2df-34bf8abf6139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:32.154470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb58399-8833-4a04-a2df-34bf8abf6139 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:32.154887 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb58399-8833-4a04-a2df-34bf8abf6139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:32.155275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb58399-8833-4a04-a2df-34bf8abf6139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:32.156278 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb58399-8833-4a04-a2df-34bf8abf6139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:32.159568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb58399-8833-4a04-a2df-34bf8abf6139 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:32.187167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1078/2128] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:27:32.193463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63731792-9e7d-4cdc-adf9-b78305d170df admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:32.219613 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63731792-9e7d-4cdc-adf9-b78305d170df admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:32.229607 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3925052-e8da-4447-9463-aeb54d2eacea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:32.230767 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3925052-e8da-4447-9463-aeb54d2eacea admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:32.231217 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3925052-e8da-4447-9463-aeb54d2eacea admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:32.231698 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3925052-e8da-4447-9463-aeb54d2eacea admin admin] PATH_INFO: `/v3/projects/37cc84ed0eb14d77b29ab260bd1ac310/users/e81f2f6439b64fa39470a0bd1dbf9847/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:32.277841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3925052-e8da-4447-9463-aeb54d2eacea admin admin] RBAC: Authorizing `identity:create_grant(project_id=37cc84ed0eb14d77b29ab260bd1ac310, user_id=e81f2f6439b64fa39470a0bd1dbf9847, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:32.277841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3925052-e8da-4447-9463-aeb54d2eacea admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:32.302750 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1051/2129] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:27:32 2022] PUT /identity/v3/projects/37cc84ed0eb14d77b29ab260bd1ac310/users/e81f2f6439b64fa39470a0bd1dbf9847/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:32.308877 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb58399-8833-4a04-a2df-34bf8abf6139 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:32.335177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb58399-8833-4a04-a2df-34bf8abf6139 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:32.561216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee953357-9644-4ca3-8113-7cebbd9ed22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:32.562145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee953357-9644-4ca3-8113-7cebbd9ed22b admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:32.563417 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee953357-9644-4ca3-8113-7cebbd9ed22b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:32.564148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee953357-9644-4ca3-8113-7cebbd9ed22b admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:32.565613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee953357-9644-4ca3-8113-7cebbd9ed22b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:32.569171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee953357-9644-4ca3-8113-7cebbd9ed22b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:32.597771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1079/2130] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:27:32.606090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3925052-e8da-4447-9463-aeb54d2eacea admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:32.651207 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3925052-e8da-4447-9463-aeb54d2eacea admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:32.673023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fed528-7772-4b22-86c5-66a7062cfe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:32.674312 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fed528-7772-4b22-86c5-66a7062cfe0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:32.674826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fed528-7772-4b22-86c5-66a7062cfe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:32.675348 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fed528-7772-4b22-86c5-66a7062cfe0b admin admin] PATH_INFO: `/v3/projects/37cc84ed0eb14d77b29ab260bd1ac310/users/e81f2f6439b64fa39470a0bd1dbf9847/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:32.709890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fed528-7772-4b22-86c5-66a7062cfe0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=37cc84ed0eb14d77b29ab260bd1ac310, user_id=e81f2f6439b64fa39470a0bd1dbf9847, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:32.717873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fed528-7772-4b22-86c5-66a7062cfe0b admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:32.748438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1052/2131] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:27:32 2022] PUT /identity/v3/projects/37cc84ed0eb14d77b29ab260bd1ac310/users/e81f2f6439b64fa39470a0bd1dbf9847/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:32.771497 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03596157-85b5-40d1-823b-3304b54323f3 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:32.772050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03596157-85b5-40d1-823b-3304b54323f3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:32.772535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03596157-85b5-40d1-823b-3304b54323f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:32.859522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-03596157-85b5-40d1-823b-3304b54323f3 None None] MFA Rules not processed for user `e81f2f6439b64fa39470a0bd1dbf9847`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:32.861704 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-03596157-85b5-40d1-823b-3304b54323f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:32.951348 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:33.000014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1080/2132] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:33.014973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fed528-7772-4b22-86c5-66a7062cfe0b admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:33.017656 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-56fed528-7772-4b22-86c5-66a7062cfe0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:33.123346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fed528-7772-4b22-86c5-66a7062cfe0b admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:33.130935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280d8f0-ae8b-4f13-a9c3-4f8d55d439ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:33.132051 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280d8f0-ae8b-4f13-a9c3-4f8d55d439ec service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:33.132574 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280d8f0-ae8b-4f13-a9c3-4f8d55d439ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:33.133097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280d8f0-ae8b-4f13-a9c3-4f8d55d439ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:33.153454 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280d8f0-ae8b-4f13-a9c3-4f8d55d439ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:33.156279 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280d8f0-ae8b-4f13-a9c3-4f8d55d439ec service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:33.176254 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1053/2133] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:27:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:27:34.038854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03596157-85b5-40d1-823b-3304b54323f3 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:34.296963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03596157-85b5-40d1-823b-3304b54323f3 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:34.313013 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e3fca-d492-479b-bf7b-8dd3296e0c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:34.320128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e3fca-d492-479b-bf7b-8dd3296e0c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:34.320128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e3fca-d492-479b-bf7b-8dd3296e0c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:34.320128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e3fca-d492-479b-bf7b-8dd3296e0c0f admin admin] PATH_INFO: `/v3/users/deecf18eaec543988ab89a960ca7885b` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:34.324911 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e3fca-d492-479b-bf7b-8dd3296e0c0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=deecf18eaec543988ab89a960ca7885b)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:34.341629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e3fca-d492-479b-bf7b-8dd3296e0c0f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:34.480853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-028e3fca-d492-479b-bf7b-8dd3296e0c0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deecf18eaec543988ab89a960ca7885b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:34.518743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-028e3fca-d492-479b-bf7b-8dd3296e0c0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deecf18eaec543988ab89a960ca7885b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:34.525881 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280d8f0-ae8b-4f13-a9c3-4f8d55d439ec service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:34.529629 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-9280d8f0-ae8b-4f13-a9c3-4f8d55d439ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:34.558390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-028e3fca-d492-479b-bf7b-8dd3296e0c0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deecf18eaec543988ab89a960ca7885b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:34.559653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-028e3fca-d492-479b-bf7b-8dd3296e0c0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deecf18eaec543988ab89a960ca7885b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:34.572113 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-028e3fca-d492-479b-bf7b-8dd3296e0c0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deecf18eaec543988ab89a960ca7885b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:34.581890 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1081/2134] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:27:34 2022] DELETE /identity/v3/users/deecf18eaec543988ab89a960ca7885b => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:34.587732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70e3a8-8933-4ac7-a35c-ec71d49e0407 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:34.588293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70e3a8-8933-4ac7-a35c-ec71d49e0407 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:34.588633 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280d8f0-ae8b-4f13-a9c3-4f8d55d439ec service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:34.588877 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70e3a8-8933-4ac7-a35c-ec71d49e0407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:34.622201 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7d70e3a8-8933-4ac7-a35c-ec71d49e0407 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:34.623991 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7d70e3a8-8933-4ac7-a35c-ec71d49e0407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:34.709857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:34.724014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1082/2135] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:34.727990 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804a340-4096-4797-bf74-9c0739d1a47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:34.728214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804a340-4096-4797-bf74-9c0739d1a47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:34.728807 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804a340-4096-4797-bf74-9c0739d1a47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:34.728967 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804a340-4096-4797-bf74-9c0739d1a47f admin admin] PATH_INFO: `/v3/users/ba49197c9a3446b68bc92c48abf57b37` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:34.731575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f804a340-4096-4797-bf74-9c0739d1a47f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba49197c9a3446b68bc92c48abf57b37)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:34.736470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f804a340-4096-4797-bf74-9c0739d1a47f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:34.739337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70e3a8-8933-4ac7-a35c-ec71d49e0407 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:34.741347 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7d70e3a8-8933-4ac7-a35c-ec71d49e0407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:34.752609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:34.823586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f804a340-4096-4797-bf74-9c0739d1a47f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba49197c9a3446b68bc92c48abf57b37'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:34.825521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f804a340-4096-4797-bf74-9c0739d1a47f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba49197c9a3446b68bc92c48abf57b37'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:34.840585 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f804a340-4096-4797-bf74-9c0739d1a47f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba49197c9a3446b68bc92c48abf57b37'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:34.866555 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f804a340-4096-4797-bf74-9c0739d1a47f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba49197c9a3446b68bc92c48abf57b37'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:34.868163 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70e3a8-8933-4ac7-a35c-ec71d49e0407 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:34.876181 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230920e-3070-4f44-9e20-99d83f4823cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:34.877402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230920e-3070-4f44-9e20-99d83f4823cc service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:34.877984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230920e-3070-4f44-9e20-99d83f4823cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:34.878600 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230920e-3070-4f44-9e20-99d83f4823cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:34.883014 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f804a340-4096-4797-bf74-9c0739d1a47f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba49197c9a3446b68bc92c48abf57b37'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:34.898543 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230920e-3070-4f44-9e20-99d83f4823cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:34.902048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230920e-3070-4f44-9e20-99d83f4823cc service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:34.913901 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1054/2136] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:27:34 2022] DELETE /identity/v3/users/ba49197c9a3446b68bc92c48abf57b37 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:34.937220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1083/2137] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:27:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:27:36.262950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804a340-4096-4797-bf74-9c0739d1a47f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:36.266790 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f804a340-4096-4797-bf74-9c0739d1a47f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:36.439971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804a340-4096-4797-bf74-9c0739d1a47f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:36.448498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163725e-2cc3-42d0-bf70-74f00282124f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:36.449688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163725e-2cc3-42d0-bf70-74f00282124f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:36.450178 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163725e-2cc3-42d0-bf70-74f00282124f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:36.450714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163725e-2cc3-42d0-bf70-74f00282124f admin admin] PATH_INFO: `/v3/projects/e5196561b7ae4adbb110ce06e81401f7` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:36.452460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163725e-2cc3-42d0-bf70-74f00282124f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5196561b7ae4adbb110ce06e81401f7)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:36.455554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163725e-2cc3-42d0-bf70-74f00282124f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:36.548821 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-9163725e-2cc3-42d0-bf70-74f00282124f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5196561b7ae4adbb110ce06e81401f7', 'request_id': 'req-9163725e-2cc3-42d0-bf70-74f00282124f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:36.558208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-9163725e-2cc3-42d0-bf70-74f00282124f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5196561b7ae4adbb110ce06e81401f7', 'request_id': 'req-9163725e-2cc3-42d0-bf70-74f00282124f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:36.568135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-9163725e-2cc3-42d0-bf70-74f00282124f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5196561b7ae4adbb110ce06e81401f7', 'request_id': 'req-9163725e-2cc3-42d0-bf70-74f00282124f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:36.598755 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-9163725e-2cc3-42d0-bf70-74f00282124f admin admin] The token cache is being invalidate because project e5196561b7ae4adbb110ce06e81401f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:27:36.599616 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-9163725e-2cc3-42d0-bf70-74f00282124f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:36.601160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-9163725e-2cc3-42d0-bf70-74f00282124f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:36.604792 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1055/2138] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:27:36 2022] DELETE /identity/v3/projects/e5196561b7ae4adbb110ce06e81401f7 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:36.611913 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edede5-b96b-49a2-811c-05ab5bb4ed11 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:36.612896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edede5-b96b-49a2-811c-05ab5bb4ed11 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:36.612896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edede5-b96b-49a2-811c-05ab5bb4ed11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:36.644506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-33edede5-b96b-49a2-811c-05ab5bb4ed11 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:36.646839 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-33edede5-b96b-49a2-811c-05ab5bb4ed11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:36.760184 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1084/2139] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:27:36.779695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6e866-c750-4304-9e5e-e9db8f7034de None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:36.780133 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6e866-c750-4304-9e5e-e9db8f7034de None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:36.780541 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6e866-c750-4304-9e5e-e9db8f7034de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:36.810468 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-faa6e866-c750-4304-9e5e-e9db8f7034de None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:36.812704 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-faa6e866-c750-4304-9e5e-e9db8f7034de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:36.827186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1056/2140] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:36.834204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edede5-b96b-49a2-811c-05ab5bb4ed11 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:36.856470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edede5-b96b-49a2-811c-05ab5bb4ed11 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:36.863249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb720f2-5cba-4fbc-9526-400dacebf078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:36.864329 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb720f2-5cba-4fbc-9526-400dacebf078 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:36.864834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb720f2-5cba-4fbc-9526-400dacebf078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:36.865363 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb720f2-5cba-4fbc-9526-400dacebf078 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:36.866604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb720f2-5cba-4fbc-9526-400dacebf078 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:27:36.867146 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb720f2-5cba-4fbc-9526-400dacebf078 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:36.871176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb720f2-5cba-4fbc-9526-400dacebf078 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:36.897002 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1085/2141] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:27:36.905254 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6e866-c750-4304-9e5e-e9db8f7034de None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:36.925110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6e866-c750-4304-9e5e-e9db8f7034de None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:36.934086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb8574-c153-4d3f-a800-04e10de0140e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:36.935065 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb8574-c153-4d3f-a800-04e10de0140e admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:36.935549 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb8574-c153-4d3f-a800-04e10de0140e admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:36.936036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb8574-c153-4d3f-a800-04e10de0140e admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:36.937305 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb8574-c153-4d3f-a800-04e10de0140e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:36.941380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb8574-c153-4d3f-a800-04e10de0140e admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:36.962866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:37.005161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1057/2142] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:27:36 2022] POST /identity/v3/projects => generated 391 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:27:37.010471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb720f2-5cba-4fbc-9526-400dacebf078 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:37.033381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb720f2-5cba-4fbc-9526-400dacebf078 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:37.181767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23563593-a9e2-4f21-a48b-8f84e4b778ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:37.183275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23563593-a9e2-4f21-a48b-8f84e4b778ac admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:37.184090 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23563593-a9e2-4f21-a48b-8f84e4b778ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:37.184596 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23563593-a9e2-4f21-a48b-8f84e4b778ac admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:37.186282 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23563593-a9e2-4f21-a48b-8f84e4b778ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:37.190731 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23563593-a9e2-4f21-a48b-8f84e4b778ac admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:37.269867 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1086/2143] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:27:37 2022] POST /identity/v3/users => generated 296 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:27:37.276161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb8574-c153-4d3f-a800-04e10de0140e admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:37.303174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb8574-c153-4d3f-a800-04e10de0140e admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:37.314591 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbd5ee-809f-49e6-8b38-2f9bcbc14b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:37.315110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbd5ee-809f-49e6-8b38-2f9bcbc14b63 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:37.315563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbd5ee-809f-49e6-8b38-2f9bcbc14b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:37.316019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbd5ee-809f-49e6-8b38-2f9bcbc14b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:37.317141 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbd5ee-809f-49e6-8b38-2f9bcbc14b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:37.319672 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbd5ee-809f-49e6-8b38-2f9bcbc14b63 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:37.328885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23563593-a9e2-4f21-a48b-8f84e4b778ac admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:37.333810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-23563593-a9e2-4f21-a48b-8f84e4b778ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:37.390882 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1058/2144] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:27:37.397773 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbd5ee-809f-49e6-8b38-2f9bcbc14b63 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:37.400373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23563593-a9e2-4f21-a48b-8f84e4b778ac admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:37.425701 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1599247-4e4a-46b0-bf77-47855a7c8f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:37.426019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbd5ee-809f-49e6-8b38-2f9bcbc14b63 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:37.426835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1599247-4e4a-46b0-bf77-47855a7c8f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:37.427371 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1599247-4e4a-46b0-bf77-47855a7c8f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:37.428098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1599247-4e4a-46b0-bf77-47855a7c8f17 admin admin] PATH_INFO: `/v3/projects/a54a028703e64d69b5f726dc027703d8` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:37.431523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1599247-4e4a-46b0-bf77-47855a7c8f17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a54a028703e64d69b5f726dc027703d8)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:37.436790 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9eb232-69f4-4f2f-91ac-d9bf2e7209d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:37.437950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9eb232-69f4-4f2f-91ac-d9bf2e7209d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:37.438469 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9eb232-69f4-4f2f-91ac-d9bf2e7209d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:37.438954 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9eb232-69f4-4f2f-91ac-d9bf2e7209d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:37.439129 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1599247-4e4a-46b0-bf77-47855a7c8f17 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:37.440306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9eb232-69f4-4f2f-91ac-d9bf2e7209d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:37.445644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9eb232-69f4-4f2f-91ac-d9bf2e7209d2 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:37.476303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1059/2145] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:27:37.479202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9eb232-69f4-4f2f-91ac-d9bf2e7209d2 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:37.503020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9eb232-69f4-4f2f-91ac-d9bf2e7209d2 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:37.513862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:37.518974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145fdd7-3600-4d7f-bca4-9589ad03a148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:37.520353 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145fdd7-3600-4d7f-bca4-9589ad03a148 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:37.522202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145fdd7-3600-4d7f-bca4-9589ad03a148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:37.523861 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145fdd7-3600-4d7f-bca4-9589ad03a148 admin admin] PATH_INFO: `/v3/projects/57cc33f50af84265ba7e0228b5d1d9ac/users/5b0e4508df8a4a44818ea86ec79d14e4/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:37.561459 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145fdd7-3600-4d7f-bca4-9589ad03a148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57cc33f50af84265ba7e0228b5d1d9ac, user_id=5b0e4508df8a4a44818ea86ec79d14e4, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:37.567698 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145fdd7-3600-4d7f-bca4-9589ad03a148 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:37.604243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1060/2146] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:27:37 2022] PUT /identity/v3/projects/57cc33f50af84265ba7e0228b5d1d9ac/users/5b0e4508df8a4a44818ea86ec79d14e4/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:37.620605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e575940-5934-4d38-ad65-660b0ca088c8 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:37.621169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e575940-5934-4d38-ad65-660b0ca088c8 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:37.621655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e575940-5934-4d38-ad65-660b0ca088c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:37.686521 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d1599247-4e4a-46b0-bf77-47855a7c8f17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a54a028703e64d69b5f726dc027703d8', 'request_id': 'req-d1599247-4e4a-46b0-bf77-47855a7c8f17'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:37.706370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d1599247-4e4a-46b0-bf77-47855a7c8f17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a54a028703e64d69b5f726dc027703d8', 'request_id': 'req-d1599247-4e4a-46b0-bf77-47855a7c8f17'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:37.724612 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d1599247-4e4a-46b0-bf77-47855a7c8f17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a54a028703e64d69b5f726dc027703d8', 'request_id': 'req-d1599247-4e4a-46b0-bf77-47855a7c8f17'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:37.727869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-9e575940-5934-4d38-ad65-660b0ca088c8 None None] MFA Rules not processed for user `5b0e4508df8a4a44818ea86ec79d14e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:37.731686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-9e575940-5934-4d38-ad65-660b0ca088c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:37.766188 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d1599247-4e4a-46b0-bf77-47855a7c8f17 admin admin] The token cache is being invalidate because project a54a028703e64d69b5f726dc027703d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:27:37.767876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d1599247-4e4a-46b0-bf77-47855a7c8f17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:37.769238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d1599247-4e4a-46b0-bf77-47855a7c8f17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:37.772595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1087/2147] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:27:37 2022] DELETE /identity/v3/projects/a54a028703e64d69b5f726dc027703d8 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:27:37.780050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1599247-4e4a-46b0-bf77-47855a7c8f17 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:37.783409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d1599247-4e4a-46b0-bf77-47855a7c8f17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:37.917443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1061/2148] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:27:37.936171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e575940-5934-4d38-ad65-660b0ca088c8 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:37.938820 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-9e575940-5934-4d38-ad65-660b0ca088c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:37.987590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1599247-4e4a-46b0-bf77-47855a7c8f17 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:37.997462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3b999-e0e5-42fd-beaa-102790a9d2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:38.000096 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3b999-e0e5-42fd-beaa-102790a9d2f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:38.001353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3b999-e0e5-42fd-beaa-102790a9d2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:38.002639 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3b999-e0e5-42fd-beaa-102790a9d2f6 admin admin] PATH_INFO: `/v3/users/e81f2f6439b64fa39470a0bd1dbf9847` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:38.006213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3b999-e0e5-42fd-beaa-102790a9d2f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e81f2f6439b64fa39470a0bd1dbf9847)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:38.011265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3b999-e0e5-42fd-beaa-102790a9d2f6 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:38.113626 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e575940-5934-4d38-ad65-660b0ca088c8 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:38.126883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d248c6e-0a63-41b3-b9a7-c9f91836c48b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:38.128583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d248c6e-0a63-41b3-b9a7-c9f91836c48b service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:38.130194 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d248c6e-0a63-41b3-b9a7-c9f91836c48b service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:38.131278 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d248c6e-0a63-41b3-b9a7-c9f91836c48b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:38.153049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5ab3b999-e0e5-42fd-beaa-102790a9d2f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e81f2f6439b64fa39470a0bd1dbf9847'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:38.173261 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d248c6e-0a63-41b3-b9a7-c9f91836c48b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:38.181582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d248c6e-0a63-41b3-b9a7-c9f91836c48b service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:38.182112 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5ab3b999-e0e5-42fd-beaa-102790a9d2f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e81f2f6439b64fa39470a0bd1dbf9847'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:38.199211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5ab3b999-e0e5-42fd-beaa-102790a9d2f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e81f2f6439b64fa39470a0bd1dbf9847'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:38.201770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5ab3b999-e0e5-42fd-beaa-102790a9d2f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e81f2f6439b64fa39470a0bd1dbf9847'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:38.219271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5ab3b999-e0e5-42fd-beaa-102790a9d2f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e81f2f6439b64fa39470a0bd1dbf9847'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:38.221021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1062/2149] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:27:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:27:38.233491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1088/2150] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:27:37 2022] DELETE /identity/v3/users/e81f2f6439b64fa39470a0bd1dbf9847 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:39.516991 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d248c6e-0a63-41b3-b9a7-c9f91836c48b service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:39.541475 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d248c6e-0a63-41b3-b9a7-c9f91836c48b service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:39.639917 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:39.718175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9aa10b-9706-43e3-affa-388a1f31bf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:39.719201 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9aa10b-9706-43e3-affa-388a1f31bf85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:39.719666 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9aa10b-9706-43e3-affa-388a1f31bf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:39.720169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9aa10b-9706-43e3-affa-388a1f31bf85 admin admin] PATH_INFO: `/v3/projects/37cc84ed0eb14d77b29ab260bd1ac310` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:39.722296 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9aa10b-9706-43e3-affa-388a1f31bf85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37cc84ed0eb14d77b29ab260bd1ac310)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:39.726174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9aa10b-9706-43e3-affa-388a1f31bf85 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:40.054950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-fb9aa10b-9706-43e3-affa-388a1f31bf85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37cc84ed0eb14d77b29ab260bd1ac310', 'request_id': 'req-fb9aa10b-9706-43e3-affa-388a1f31bf85'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:40.078617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-fb9aa10b-9706-43e3-affa-388a1f31bf85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37cc84ed0eb14d77b29ab260bd1ac310', 'request_id': 'req-fb9aa10b-9706-43e3-affa-388a1f31bf85'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:40.113338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-fb9aa10b-9706-43e3-affa-388a1f31bf85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37cc84ed0eb14d77b29ab260bd1ac310', 'request_id': 'req-fb9aa10b-9706-43e3-affa-388a1f31bf85'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:40.163843 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-fb9aa10b-9706-43e3-affa-388a1f31bf85 admin admin] The token cache is being invalidate because project 37cc84ed0eb14d77b29ab260bd1ac310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:27:40.164417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-fb9aa10b-9706-43e3-affa-388a1f31bf85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:40.165638 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-fb9aa10b-9706-43e3-affa-388a1f31bf85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:40.171282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1063/2151] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:27:39 2022] DELETE /identity/v3/projects/37cc84ed0eb14d77b29ab260bd1ac310 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:40.181904 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547dec3-c97c-47bd-a6a6-d347146d9701 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:40.182671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547dec3-c97c-47bd-a6a6-d347146d9701 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:40.182671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547dec3-c97c-47bd-a6a6-d347146d9701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:40.228696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e547dec3-c97c-47bd-a6a6-d347146d9701 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:40.230261 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e547dec3-c97c-47bd-a6a6-d347146d9701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:40.278302 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:40.498327 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1089/2152] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:40.502036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b739ac-fb85-4618-8f4e-20d0a9142a89 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:40.502557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b739ac-fb85-4618-8f4e-20d0a9142a89 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:40.502852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b739ac-fb85-4618-8f4e-20d0a9142a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:40.562075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-c1b739ac-fb85-4618-8f4e-20d0a9142a89 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:40.568404 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c1b739ac-fb85-4618-8f4e-20d0a9142a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:40.596811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1064/2153] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:27:40.602268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547dec3-c97c-47bd-a6a6-d347146d9701 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:40.632204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547dec3-c97c-47bd-a6a6-d347146d9701 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:40.648912 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef93f3-2609-49cf-a19e-a9534dfb407a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:40.652483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef93f3-2609-49cf-a19e-a9534dfb407a admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:40.653476 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef93f3-2609-49cf-a19e-a9534dfb407a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:40.654375 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef93f3-2609-49cf-a19e-a9534dfb407a admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:40.656516 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef93f3-2609-49cf-a19e-a9534dfb407a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:27:40.657797 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef93f3-2609-49cf-a19e-a9534dfb407a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:40.663182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef93f3-2609-49cf-a19e-a9534dfb407a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:40.707735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1090/2154] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:27:40.715221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b739ac-fb85-4618-8f4e-20d0a9142a89 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:40.732927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b739ac-fb85-4618-8f4e-20d0a9142a89 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:40.743191 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a1b95-b4d4-42ae-b10e-b1597ec218b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:40.746431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a1b95-b4d4-42ae-b10e-b1597ec218b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:40.747197 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a1b95-b4d4-42ae-b10e-b1597ec218b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:40.747675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a1b95-b4d4-42ae-b10e-b1597ec218b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:40.750581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a1b95-b4d4-42ae-b10e-b1597ec218b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:40.754867 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a1b95-b4d4-42ae-b10e-b1597ec218b8 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:40.887460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1065/2155] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:27:40 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:27:40.901554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef93f3-2609-49cf-a19e-a9534dfb407a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:40.954416 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef93f3-2609-49cf-a19e-a9534dfb407a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:41.125439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b3770-0579-42b2-9f3f-791cdb959d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:41.126340 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b3770-0579-42b2-9f3f-791cdb959d80 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:41.126698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b3770-0579-42b2-9f3f-791cdb959d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:41.127148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b3770-0579-42b2-9f3f-791cdb959d80 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:41.128233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b3770-0579-42b2-9f3f-791cdb959d80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:41.131864 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b3770-0579-42b2-9f3f-791cdb959d80 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:41.177911 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1091/2156] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:27:40 2022] POST /identity/v3/users => generated 292 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:27:41.184825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a1b95-b4d4-42ae-b10e-b1597ec218b8 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:41.210260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a1b95-b4d4-42ae-b10e-b1597ec218b8 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:41.219295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7045f5-08f8-4318-8f8e-a736a123d4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:41.220225 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7045f5-08f8-4318-8f8e-a736a123d4ea admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:41.220583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7045f5-08f8-4318-8f8e-a736a123d4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:41.220924 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7045f5-08f8-4318-8f8e-a736a123d4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:41.221775 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7045f5-08f8-4318-8f8e-a736a123d4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:41.224079 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7045f5-08f8-4318-8f8e-a736a123d4ea admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:41.246602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1066/2157] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:27:41.253681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b3770-0579-42b2-9f3f-791cdb959d80 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:41.273294 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b3770-0579-42b2-9f3f-791cdb959d80 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:41.287082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77872407-bf7c-44e7-80de-977cededcd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:41.288280 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77872407-bf7c-44e7-80de-977cededcd41 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:41.288911 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77872407-bf7c-44e7-80de-977cededcd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:41.289920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77872407-bf7c-44e7-80de-977cededcd41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:41.291305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77872407-bf7c-44e7-80de-977cededcd41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:41.293953 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77872407-bf7c-44e7-80de-977cededcd41 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:41.321145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1092/2158] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:27:41.327208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7045f5-08f8-4318-8f8e-a736a123d4ea admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:41.348105 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7045f5-08f8-4318-8f8e-a736a123d4ea admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:41.370566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701a39a-8037-4e5c-8939-d3663d6a05f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:41.371761 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701a39a-8037-4e5c-8939-d3663d6a05f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:41.372229 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701a39a-8037-4e5c-8939-d3663d6a05f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:41.372869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701a39a-8037-4e5c-8939-d3663d6a05f8 admin admin] PATH_INFO: `/v3/projects/bf8e9d006f1e4a0aa7109e652e8d001f/users/3ecdf4bba4204bc988fe116007282323/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:41.405617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701a39a-8037-4e5c-8939-d3663d6a05f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf8e9d006f1e4a0aa7109e652e8d001f, user_id=3ecdf4bba4204bc988fe116007282323, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:41.411120 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701a39a-8037-4e5c-8939-d3663d6a05f8 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:41.538716 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1067/2159] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:27:41 2022] PUT /identity/v3/projects/bf8e9d006f1e4a0aa7109e652e8d001f/users/3ecdf4bba4204bc988fe116007282323/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:41.545180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc12a9b-2cde-4b97-b5ff-9e2c5f0d21d4 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:41.545591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc12a9b-2cde-4b97-b5ff-9e2c5f0d21d4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:41.546077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc12a9b-2cde-4b97-b5ff-9e2c5f0d21d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:41.602732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-bcc12a9b-2cde-4b97-b5ff-9e2c5f0d21d4 None None] MFA Rules not processed for user `3ecdf4bba4204bc988fe116007282323`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:41.604912 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bcc12a9b-2cde-4b97-b5ff-9e2c5f0d21d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:41.762450 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1093/2160] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:27:41.765329 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701a39a-8037-4e5c-8939-d3663d6a05f8 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:41.954031 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701a39a-8037-4e5c-8939-d3663d6a05f8 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:41.968741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b646a8-e976-44b0-9c78-de69e5dc2a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:41.970092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b646a8-e976-44b0-9c78-de69e5dc2a28 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:41.970638 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b646a8-e976-44b0-9c78-de69e5dc2a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:41.971064 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b646a8-e976-44b0-9c78-de69e5dc2a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:41.972305 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b646a8-e976-44b0-9c78-de69e5dc2a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:41.976748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b646a8-e976-44b0-9c78-de69e5dc2a28 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:41.987819 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:42.052122 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1068/2161] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:27:41 2022] POST /identity/v3/projects => generated 383 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:27:42.057267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc12a9b-2cde-4b97-b5ff-9e2c5f0d21d4 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:42.080303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc12a9b-2cde-4b97-b5ff-9e2c5f0d21d4 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:42.118637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:42.286964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1929a-6b9c-43b4-8e38-35f31654ce71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:42.288071 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1929a-6b9c-43b4-8e38-35f31654ce71 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:42.288726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1929a-6b9c-43b4-8e38-35f31654ce71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:42.289363 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1929a-6b9c-43b4-8e38-35f31654ce71 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:42.290596 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1929a-6b9c-43b4-8e38-35f31654ce71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:42.296080 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1929a-6b9c-43b4-8e38-35f31654ce71 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:42.349177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1094/2162] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:27:42 2022] POST /identity/v3/users => generated 292 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:27:42.354082 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b646a8-e976-44b0-9c78-de69e5dc2a28 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:42.375258 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b646a8-e976-44b0-9c78-de69e5dc2a28 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:42.385323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83b157-6e1e-4c74-9ef5-a3158e12db01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:42.391750 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83b157-6e1e-4c74-9ef5-a3158e12db01 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:42.392245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83b157-6e1e-4c74-9ef5-a3158e12db01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:42.392744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83b157-6e1e-4c74-9ef5-a3158e12db01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:42.394250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83b157-6e1e-4c74-9ef5-a3158e12db01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:42.397318 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83b157-6e1e-4c74-9ef5-a3158e12db01 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:42.431852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1069/2163] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:27:42.437855 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1929a-6b9c-43b4-8e38-35f31654ce71 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:42.488999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1929a-6b9c-43b4-8e38-35f31654ce71 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:42.500208 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b6730-89f9-4aa3-a367-045e6aa0c839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:42.501524 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b6730-89f9-4aa3-a367-045e6aa0c839 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:42.502125 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b6730-89f9-4aa3-a367-045e6aa0c839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:42.503002 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b6730-89f9-4aa3-a367-045e6aa0c839 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:42.504380 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b6730-89f9-4aa3-a367-045e6aa0c839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:42.508205 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b6730-89f9-4aa3-a367-045e6aa0c839 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:42.533893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1095/2164] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:27:42.542208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83b157-6e1e-4c74-9ef5-a3158e12db01 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:42.564167 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83b157-6e1e-4c74-9ef5-a3158e12db01 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:42.575200 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008576ec-0a68-4a04-a3fb-762ba7f244f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:42.577598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008576ec-0a68-4a04-a3fb-762ba7f244f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:42.578417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008576ec-0a68-4a04-a3fb-762ba7f244f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:42.579380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008576ec-0a68-4a04-a3fb-762ba7f244f9 admin admin] PATH_INFO: `/v3/projects/bd3dc6535b2e4f9aac0568fc20be2739/users/91df845ca20b483c930ef07e0af54d02/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:42.614553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008576ec-0a68-4a04-a3fb-762ba7f244f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd3dc6535b2e4f9aac0568fc20be2739, user_id=91df845ca20b483c930ef07e0af54d02, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:42.621769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008576ec-0a68-4a04-a3fb-762ba7f244f9 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:42.666874 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1070/2165] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:27:42 2022] PUT /identity/v3/projects/bd3dc6535b2e4f9aac0568fc20be2739/users/91df845ca20b483c930ef07e0af54d02/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:27:42.687459 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e54be-1d85-4a2a-b5a0-6fa402a2beb6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:42.689649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e54be-1d85-4a2a-b5a0-6fa402a2beb6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:42.690218 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e54be-1d85-4a2a-b5a0-6fa402a2beb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:42.763379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-3e4e54be-1d85-4a2a-b5a0-6fa402a2beb6 None None] MFA Rules not processed for user `91df845ca20b483c930ef07e0af54d02`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:42.765722 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3e4e54be-1d85-4a2a-b5a0-6fa402a2beb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:43.009748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1096/2166] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:43.017679 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008576ec-0a68-4a04-a3fb-762ba7f244f9 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:43.140995 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd9975-8bdb-4fb7-9490-e92716bed592 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:43.142127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd9975-8bdb-4fb7-9490-e92716bed592 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:43.142871 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd9975-8bdb-4fb7-9490-e92716bed592 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:43.146567 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1097/2167] 10.222.0.88 () {54 vars in 921 bytes} [Tue Aug 23 22:27:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:27:43.159518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5175a-c4d1-4b80-a68d-3675b6b0297f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:43.160041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5175a-c4d1-4b80-a68d-3675b6b0297f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:43.160182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5175a-c4d1-4b80-a68d-3675b6b0297f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:43.198784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0df5175a-c4d1-4b80-a68d-3675b6b0297f None None] MFA Rules not processed for user `e3517b3b077a4a1fa76163657b46930c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:43.202487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0df5175a-c4d1-4b80-a68d-3675b6b0297f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:43.262894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008576ec-0a68-4a04-a3fb-762ba7f244f9 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:43.272793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6e0f8-733e-46d8-8118-a0cec53efcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:43.274769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6e0f8-733e-46d8-8118-a0cec53efcb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:43.275703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6e0f8-733e-46d8-8118-a0cec53efcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:43.276316 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6e0f8-733e-46d8-8118-a0cec53efcb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:43.277410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6e0f8-733e-46d8-8118-a0cec53efcb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:43.280413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6e0f8-733e-46d8-8118-a0cec53efcb2 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:43.345394 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1071/2168] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:27:43 2022] POST /identity/v3/projects => generated 385 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:27:43.351909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6e0f8-733e-46d8-8118-a0cec53efcb2 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:43.446599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1098/2169] 10.222.0.88 () {58 vars in 1018 bytes} [Tue Aug 23 22:27:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:27:43.481021 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5175a-c4d1-4b80-a68d-3675b6b0297f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:43.487078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0df5175a-c4d1-4b80-a68d-3675b6b0297f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:43.524605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6e0f8-733e-46d8-8118-a0cec53efcb2 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:43.532770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9149e-0f2a-4538-b48d-56c9e4fc8804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:43.533931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9149e-0f2a-4538-b48d-56c9e4fc8804 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:43.534496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9149e-0f2a-4538-b48d-56c9e4fc8804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:43.534953 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9149e-0f2a-4538-b48d-56c9e4fc8804 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:43.536074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:43.537140 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9149e-0f2a-4538-b48d-56c9e4fc8804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:43.553420 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9149e-0f2a-4538-b48d-56c9e4fc8804 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:43.597669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1072/2170] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:27:43 2022] POST /identity/v3/users => generated 299 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:27:43.604479 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9149e-0f2a-4538-b48d-56c9e4fc8804 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:43.615743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5175a-c4d1-4b80-a68d-3675b6b0297f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:43.623443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9149e-0f2a-4538-b48d-56c9e4fc8804 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:43.634194 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812efba1-4d2b-4f23-b3d6-c91dd39a6b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:43.635241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812efba1-4d2b-4f23-b3d6-c91dd39a6b02 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:43.635739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812efba1-4d2b-4f23-b3d6-c91dd39a6b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:43.636221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812efba1-4d2b-4f23-b3d6-c91dd39a6b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:43.637789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812efba1-4d2b-4f23-b3d6-c91dd39a6b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:43.641623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812efba1-4d2b-4f23-b3d6-c91dd39a6b02 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:43.669243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1073/2171] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:27:43.674183 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812efba1-4d2b-4f23-b3d6-c91dd39a6b02 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:43.696240 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812efba1-4d2b-4f23-b3d6-c91dd39a6b02 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:43.706076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127b403-f5da-4950-a5a3-8ae1b14ee3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:43.707447 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127b403-f5da-4950-a5a3-8ae1b14ee3d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:43.708198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127b403-f5da-4950-a5a3-8ae1b14ee3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:43.708962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127b403-f5da-4950-a5a3-8ae1b14ee3d4 admin admin] PATH_INFO: `/v3/projects/a805f131a3934c4bad598249d712af81/users/9a8d693a13e44f2787612dd1499cea0a/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:43.734420 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:43.738567 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c8dae-bb49-460e-b049-33379a199c57 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2211e237fe554a82baf129e440a0fb43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:43.739077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c8dae-bb49-460e-b049-33379a199c57 service glance] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:43.739575 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c8dae-bb49-460e-b049-33379a199c57 service glance] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:43.740066 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c8dae-bb49-460e-b049-33379a199c57 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:43.740762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127b403-f5da-4950-a5a3-8ae1b14ee3d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a805f131a3934c4bad598249d712af81, user_id=9a8d693a13e44f2787612dd1499cea0a, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:43.743936 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127b403-f5da-4950-a5a3-8ae1b14ee3d4 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:43.765139 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1074/2172] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:27:43 2022] PUT /identity/v3/projects/a805f131a3934c4bad598249d712af81/users/9a8d693a13e44f2787612dd1499cea0a/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:43.769806 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127b403-f5da-4950-a5a3-8ae1b14ee3d4 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:43.904746 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127b403-f5da-4950-a5a3-8ae1b14ee3d4 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:43.914224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c8dae-bb49-460e-b049-33379a199c57 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:43.916085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20da8a2-a81a-4ff7-b35b-ba483163f19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:43.916887 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c8dae-bb49-460e-b049-33379a199c57 service glance] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:43.917653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20da8a2-a81a-4ff7-b35b-ba483163f19c admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:43.918306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20da8a2-a81a-4ff7-b35b-ba483163f19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:43.918952 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20da8a2-a81a-4ff7-b35b-ba483163f19c admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:43.920989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20da8a2-a81a-4ff7-b35b-ba483163f19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:43.925215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20da8a2-a81a-4ff7-b35b-ba483163f19c admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:43.942697 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1099/2173] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:27:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:27:43.953472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1075/2174] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:27:43.960519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c8dae-bb49-460e-b049-33379a199c57 service glance] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:43.985094 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c8dae-bb49-460e-b049-33379a199c57 service glance] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:43.995691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6542e-91c5-4564-b69d-930f6b36bb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:43.996077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6542e-91c5-4564-b69d-930f6b36bb45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:43.996694 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6542e-91c5-4564-b69d-930f6b36bb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:43.997150 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6542e-91c5-4564-b69d-930f6b36bb45 admin admin] PATH_INFO: `/v3/projects/a805f131a3934c4bad598249d712af81/users/9a8d693a13e44f2787612dd1499cea0a/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:44.000840 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6542e-91c5-4564-b69d-930f6b36bb45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a805f131a3934c4bad598249d712af81, user_id=9a8d693a13e44f2787612dd1499cea0a, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:44.006560 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6542e-91c5-4564-b69d-930f6b36bb45 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:44.039227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1100/2175] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:27:43 2022] PUT /identity/v3/projects/a805f131a3934c4bad598249d712af81/users/9a8d693a13e44f2787612dd1499cea0a/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:44.072833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1f08b-60d3-46e9-8d7d-56c2c039a83d None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:44.073869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1f08b-60d3-46e9-8d7d-56c2c039a83d None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:44.074638 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1f08b-60d3-46e9-8d7d-56c2c039a83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:44.125394 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e08b43-e404-4704-b5dd-a7c24422865f None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:44.126761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e08b43-e404-4704-b5dd-a7c24422865f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:44.127614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e08b43-e404-4704-b5dd-a7c24422865f None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:44.130431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1101/2176] 10.222.0.88 () {54 vars in 921 bytes} [Tue Aug 23 22:27:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:27:44.139724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-1ca1f08b-60d3-46e9-8d7d-56c2c039a83d None None] MFA Rules not processed for user `9a8d693a13e44f2787612dd1499cea0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:44.140693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91087883-c54b-41f1-b4ec-faef4039cfa2 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:44.141352 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-1ca1f08b-60d3-46e9-8d7d-56c2c039a83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:44.142408 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91087883-c54b-41f1-b4ec-faef4039cfa2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:44.143511 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91087883-c54b-41f1-b4ec-faef4039cfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:44.195055 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-91087883-c54b-41f1-b4ec-faef4039cfa2 None None] MFA Rules not processed for user `6505100494d4496591407c48b48535ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:44.198774 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-91087883-c54b-41f1-b4ec-faef4039cfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:44.323971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1076/2177] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:44.392601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1102/2178] 10.222.0.88 () {58 vars in 1018 bytes} [Tue Aug 23 22:27:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:27:44.419098 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1f08b-60d3-46e9-8d7d-56c2c039a83d None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:44.421407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-1ca1f08b-60d3-46e9-8d7d-56c2c039a83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:44.452864 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1f08b-60d3-46e9-8d7d-56c2c039a83d None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:44.463687 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1a649-0681-48d9-9f7d-407518e1bf03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:44.465716 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1a649-0681-48d9-9f7d-407518e1bf03 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:44.466955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1a649-0681-48d9-9f7d-407518e1bf03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:44.468100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1a649-0681-48d9-9f7d-407518e1bf03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:44.488751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1a649-0681-48d9-9f7d-407518e1bf03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:44.495056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1a649-0681-48d9-9f7d-407518e1bf03 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:44.534833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1077/2179] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:27:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:27:44.678717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91087883-c54b-41f1-b4ec-faef4039cfa2 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:44.682604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-91087883-c54b-41f1-b4ec-faef4039cfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:44.710124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:44.864618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b31e22-5395-4ab6-a7d2-49f3749ad3a8 None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:44.865704 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b31e22-5395-4ab6-a7d2-49f3749ad3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:44.866496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b31e22-5395-4ab6-a7d2-49f3749ad3a8 None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:44.870474 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1078/2180] 10.222.0.88 () {54 vars in 921 bytes} [Tue Aug 23 22:27:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:27:44.891171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2136c3-eaed-43f6-a871-94fd8a5101db None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:44.892423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2136c3-eaed-43f6-a871-94fd8a5101db None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:44.893195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2136c3-eaed-43f6-a871-94fd8a5101db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:44.938647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91087883-c54b-41f1-b4ec-faef4039cfa2 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:44.944732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c4574-19a2-41f9-9b83-e09151b59670 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:44.946291 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c4574-19a2-41f9-9b83-e09151b59670 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:44.947158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c4574-19a2-41f9-9b83-e09151b59670 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:44.947656 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c4574-19a2-41f9-9b83-e09151b59670 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:45.011601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-0b2136c3-eaed-43f6-a871-94fd8a5101db None None] MFA Rules not processed for user `e3517b3b077a4a1fa76163657b46930c`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:45.014220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0b2136c3-eaed-43f6-a871-94fd8a5101db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:45.087775 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c4574-19a2-41f9-9b83-e09151b59670 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:45.091092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c4574-19a2-41f9-9b83-e09151b59670 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:45.111118 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1103/2181] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:27:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:27:45.125936 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:45.165748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1079/2182] 10.222.0.88 () {58 vars in 1018 bytes} [Tue Aug 23 22:27:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:27:45.184386 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c4574-19a2-41f9-9b83-e09151b59670 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:45.208615 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c4574-19a2-41f9-9b83-e09151b59670 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:45.230738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2108180-dcb4-4dac-8927-4d8007e28ece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:45.231669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2108180-dcb4-4dac-8927-4d8007e28ece service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:45.232154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2108180-dcb4-4dac-8927-4d8007e28ece service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:45.232617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2108180-dcb4-4dac-8927-4d8007e28ece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:45.282868 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2108180-dcb4-4dac-8927-4d8007e28ece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:45.286179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2108180-dcb4-4dac-8927-4d8007e28ece service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:45.322080 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1104/2183] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:27:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:27:49.579665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2136c3-eaed-43f6-a871-94fd8a5101db None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:49.599085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2136c3-eaed-43f6-a871-94fd8a5101db None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:49.606968 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea21d0-c610-4c89-8bcd-d643a87a57f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:49.608070 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea21d0-c610-4c89-8bcd-d643a87a57f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:49.608624 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea21d0-c610-4c89-8bcd-d643a87a57f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:49.609259 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea21d0-c610-4c89-8bcd-d643a87a57f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:49.762831 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea21d0-c610-4c89-8bcd-d643a87a57f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:49.765737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea21d0-c610-4c89-8bcd-d643a87a57f9 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:49.787011 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1080/2184] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:27:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:27:52.376176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2108180-dcb4-4dac-8927-4d8007e28ece service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:52.391604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2108180-dcb4-4dac-8927-4d8007e28ece service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:52.398123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8273e-e64e-4157-af8a-afe0c70943c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:52.399629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8273e-e64e-4157-af8a-afe0c70943c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:52.400144 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8273e-e64e-4157-af8a-afe0c70943c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:52.403620 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8273e-e64e-4157-af8a-afe0c70943c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:52.419851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8273e-e64e-4157-af8a-afe0c70943c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:52.421706 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8273e-e64e-4157-af8a-afe0c70943c6 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:52.435024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1105/2185] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:27:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:27:56.938344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea21d0-c610-4c89-8bcd-d643a87a57f9 service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:56.966441 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea21d0-c610-4c89-8bcd-d643a87a57f9 service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:56.973384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74031f2f-2017-49f5-ad7a-f7cc723043c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:56.974493 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74031f2f-2017-49f5-ad7a-f7cc723043c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:56.975004 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74031f2f-2017-49f5-ad7a-f7cc723043c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:56.975497 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74031f2f-2017-49f5-ad7a-f7cc723043c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:56.978781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-74031f2f-2017-49f5-ad7a-f7cc723043c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:57.102143 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74031f2f-2017-49f5-ad7a-f7cc723043c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:57.103933 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74031f2f-2017-49f5-ad7a-f7cc723043c7 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:57.128243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1081/2186] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:27:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:27:57.247314 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8273e-e64e-4157-af8a-afe0c70943c6 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:57.466023 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8273e-e64e-4157-af8a-afe0c70943c6 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:57.475691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c13c69-210a-4230-af43-c4b1d393e5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:57.476920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c13c69-210a-4230-af43-c4b1d393e5c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:57.477464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c13c69-210a-4230-af43-c4b1d393e5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:57.477909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c13c69-210a-4230-af43-c4b1d393e5c7 admin admin] PATH_INFO: `/v3/users/3ecdf4bba4204bc988fe116007282323` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:57.479966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c13c69-210a-4230-af43-c4b1d393e5c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ecdf4bba4204bc988fe116007282323)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:57.482827 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c13c69-210a-4230-af43-c4b1d393e5c7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:57.621879 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:57.648489 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-64c13c69-210a-4230-af43-c4b1d393e5c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ecdf4bba4204bc988fe116007282323'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:57.701695 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-64c13c69-210a-4230-af43-c4b1d393e5c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ecdf4bba4204bc988fe116007282323'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:57.746523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-64c13c69-210a-4230-af43-c4b1d393e5c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ecdf4bba4204bc988fe116007282323'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:57.751290 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-64c13c69-210a-4230-af43-c4b1d393e5c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ecdf4bba4204bc988fe116007282323'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:57.768089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-64c13c69-210a-4230-af43-c4b1d393e5c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ecdf4bba4204bc988fe116007282323'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:57.803988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1106/2187] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:27:57 2022] DELETE /identity/v3/users/3ecdf4bba4204bc988fe116007282323 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:57.820663 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3e29e-6a0e-4bc8-b2cd-45ffa3829cd7 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:57.821696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3e29e-6a0e-4bc8-b2cd-45ffa3829cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:57.824382 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3e29e-6a0e-4bc8-b2cd-45ffa3829cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:57.883809 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-86a3e29e-6a0e-4bc8-b2cd-45ffa3829cd7 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:57.886690 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-86a3e29e-6a0e-4bc8-b2cd-45ffa3829cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:58.151342 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1082/2188] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:58.172496 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c13c69-210a-4230-af43-c4b1d393e5c7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:58.181033 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-64c13c69-210a-4230-af43-c4b1d393e5c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:58.441947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c13c69-210a-4230-af43-c4b1d393e5c7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:58.460547 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ec06c-ebe2-4a32-b249-bbb130eaf623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:58.465879 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ec06c-ebe2-4a32-b249-bbb130eaf623 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:58.465879 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ec06c-ebe2-4a32-b249-bbb130eaf623 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:58.469080 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ec06c-ebe2-4a32-b249-bbb130eaf623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:58.520722 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ec06c-ebe2-4a32-b249-bbb130eaf623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:58.525141 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ec06c-ebe2-4a32-b249-bbb130eaf623 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:58.569370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1107/2189] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:27:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:27:59.754379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3e29e-6a0e-4bc8-b2cd-45ffa3829cd7 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:59.755531 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-86a3e29e-6a0e-4bc8-b2cd-45ffa3829cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:59.781132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3e29e-6a0e-4bc8-b2cd-45ffa3829cd7 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:59.797310 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a52620-2866-403b-8c05-38367f527e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:59.802384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a52620-2866-403b-8c05-38367f527e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:59.802948 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a52620-2866-403b-8c05-38367f527e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:59.812631 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a52620-2866-403b-8c05-38367f527e09 admin admin] PATH_INFO: `/v3/users/5b0e4508df8a4a44818ea86ec79d14e4` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:59.832319 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a52620-2866-403b-8c05-38367f527e09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b0e4508df8a4a44818ea86ec79d14e4)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:59.841377 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a52620-2866-403b-8c05-38367f527e09 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:59.939356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ec06c-ebe2-4a32-b249-bbb130eaf623 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:59.942085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bd7ec06c-ebe2-4a32-b249-bbb130eaf623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:59.951631 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:59.985337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ec06c-ebe2-4a32-b249-bbb130eaf623 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:59.996637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:59.998650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:59.999556 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:00.000338 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8 admin admin] PATH_INFO: `/v3/projects/bf8e9d006f1e4a0aa7109e652e8d001f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:00.003106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf8e9d006f1e4a0aa7109e652e8d001f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:00.008583 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:00.103466 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e4a52620-2866-403b-8c05-38367f527e09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b0e4508df8a4a44818ea86ec79d14e4'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:00.107925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e4a52620-2866-403b-8c05-38367f527e09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b0e4508df8a4a44818ea86ec79d14e4'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:00.150381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e4a52620-2866-403b-8c05-38367f527e09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b0e4508df8a4a44818ea86ec79d14e4'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:00.194204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e4a52620-2866-403b-8c05-38367f527e09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b0e4508df8a4a44818ea86ec79d14e4'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:00.251775 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e4a52620-2866-403b-8c05-38367f527e09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b0e4508df8a4a44818ea86ec79d14e4'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:00.279685 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf8e9d006f1e4a0aa7109e652e8d001f', 'request_id': 'req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:00.313194 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1083/2190] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:27:59 2022] DELETE /identity/v3/users/5b0e4508df8a4a44818ea86ec79d14e4 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:28:00.317321 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e24b42-d1af-455c-950c-1e640ad7ff4c None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:00.317834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e24b42-d1af-455c-950c-1e640ad7ff4c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:00.318340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e24b42-d1af-455c-950c-1e640ad7ff4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:00.332866 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf8e9d006f1e4a0aa7109e652e8d001f', 'request_id': 'req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:00.357966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf8e9d006f1e4a0aa7109e652e8d001f', 'request_id': 'req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:00.395716 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-75e24b42-d1af-455c-950c-1e640ad7ff4c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:28:00.399221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-75e24b42-d1af-455c-950c-1e640ad7ff4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:00.410145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8 admin admin] The token cache is being invalidate because project bf8e9d006f1e4a0aa7109e652e8d001f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:28:00.414719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:00.416071 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:00.427362 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1108/2191] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:27:59 2022] DELETE /identity/v3/projects/bf8e9d006f1e4a0aa7109e652e8d001f => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:28:00.438989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:00.445878 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:00.653138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1084/2192] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:28:00.657674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:28:00.678628 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e24b42-d1af-455c-950c-1e640ad7ff4c None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:00.683031 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-75e24b42-d1af-455c-950c-1e640ad7ff4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:00.728650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b3024-9e6a-4497-9abe-7b5bcbf2ebf8 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:00.739957 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccc626-f523-4cce-b7c7-2cb72cee4934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:00.744161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccc626-f523-4cce-b7c7-2cb72cee4934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:00.744688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccc626-f523-4cce-b7c7-2cb72cee4934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:00.746097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccc626-f523-4cce-b7c7-2cb72cee4934 admin admin] PATH_INFO: `/v3/users/91df845ca20b483c930ef07e0af54d02` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:00.747968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccc626-f523-4cce-b7c7-2cb72cee4934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91df845ca20b483c930ef07e0af54d02)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:00.755097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccc626-f523-4cce-b7c7-2cb72cee4934 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:00.940473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-47ccc626-f523-4cce-b7c7-2cb72cee4934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91df845ca20b483c930ef07e0af54d02'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:00.956253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e24b42-d1af-455c-950c-1e640ad7ff4c None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:00.976095 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc86f8-fce8-4ff7-8406-6df3e30db3b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:00.977612 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc86f8-fce8-4ff7-8406-6df3e30db3b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:00.978014 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc86f8-fce8-4ff7-8406-6df3e30db3b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:00.978502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc86f8-fce8-4ff7-8406-6df3e30db3b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:00.980160 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-47ccc626-f523-4cce-b7c7-2cb72cee4934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91df845ca20b483c930ef07e0af54d02'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:01.003945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc86f8-fce8-4ff7-8406-6df3e30db3b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:01.011019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc86f8-fce8-4ff7-8406-6df3e30db3b0 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:01.017732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-47ccc626-f523-4cce-b7c7-2cb72cee4934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91df845ca20b483c930ef07e0af54d02'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:01.019987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-47ccc626-f523-4cce-b7c7-2cb72cee4934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91df845ca20b483c930ef07e0af54d02'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:01.044880 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-47ccc626-f523-4cce-b7c7-2cb72cee4934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91df845ca20b483c930ef07e0af54d02'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:01.051142 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1085/2193] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:28:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:28:01.065743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1109/2194] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:28:00 2022] DELETE /identity/v3/users/91df845ca20b483c930ef07e0af54d02 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:28:02.901642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc86f8-fce8-4ff7-8406-6df3e30db3b0 service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:02.905440 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c6cc86f8-fce8-4ff7-8406-6df3e30db3b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:03.014291 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc86f8-fce8-4ff7-8406-6df3e30db3b0 service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:03.214078 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:28:03.298668 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040f400-7a6e-4335-a475-12b2f429d28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:03.299804 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040f400-7a6e-4335-a475-12b2f429d28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:03.300113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040f400-7a6e-4335-a475-12b2f429d28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:03.300683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040f400-7a6e-4335-a475-12b2f429d28d admin admin] PATH_INFO: `/v3/projects/57cc33f50af84265ba7e0228b5d1d9ac` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:03.303397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040f400-7a6e-4335-a475-12b2f429d28d admin admin] RBAC: Authorizing `identity:delete_project(project_id=57cc33f50af84265ba7e0228b5d1d9ac)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:03.308939 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040f400-7a6e-4335-a475-12b2f429d28d admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:03.456623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d040f400-7a6e-4335-a475-12b2f429d28d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57cc33f50af84265ba7e0228b5d1d9ac', 'request_id': 'req-d040f400-7a6e-4335-a475-12b2f429d28d'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:03.479001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d040f400-7a6e-4335-a475-12b2f429d28d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57cc33f50af84265ba7e0228b5d1d9ac', 'request_id': 'req-d040f400-7a6e-4335-a475-12b2f429d28d'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:03.492817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccc626-f523-4cce-b7c7-2cb72cee4934 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:03.497349 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-47ccc626-f523-4cce-b7c7-2cb72cee4934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:03.499548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d040f400-7a6e-4335-a475-12b2f429d28d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57cc33f50af84265ba7e0228b5d1d9ac', 'request_id': 'req-d040f400-7a6e-4335-a475-12b2f429d28d'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:03.553739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d040f400-7a6e-4335-a475-12b2f429d28d admin admin] The token cache is being invalidate because project 57cc33f50af84265ba7e0228b5d1d9ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:28:03.557675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d040f400-7a6e-4335-a475-12b2f429d28d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:03.560160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d040f400-7a6e-4335-a475-12b2f429d28d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:03.563240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccc626-f523-4cce-b7c7-2cb72cee4934 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:03.569357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1086/2195] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:28:02 2022] DELETE /identity/v3/projects/57cc33f50af84265ba7e0228b5d1d9ac => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:28:03.808914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96deda92-9ffb-4134-9427-e62de8117ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:03.810153 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96deda92-9ffb-4134-9427-e62de8117ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:03.810793 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96deda92-9ffb-4134-9427-e62de8117ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:03.811221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96deda92-9ffb-4134-9427-e62de8117ea0 admin admin] PATH_INFO: `/v3/projects/bd3dc6535b2e4f9aac0568fc20be2739` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:03.815486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96deda92-9ffb-4134-9427-e62de8117ea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd3dc6535b2e4f9aac0568fc20be2739)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:03.818272 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa7850-104d-4be6-9792-73807fe3d6fc None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:03.820540 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa7850-104d-4be6-9792-73807fe3d6fc None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:03.822086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa7850-104d-4be6-9792-73807fe3d6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:03.823628 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96deda92-9ffb-4134-9427-e62de8117ea0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:03.905843 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-87aa7850-104d-4be6-9792-73807fe3d6fc None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:28:03.916290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-87aa7850-104d-4be6-9792-73807fe3d6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:04.025377 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-96deda92-9ffb-4134-9427-e62de8117ea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd3dc6535b2e4f9aac0568fc20be2739', 'request_id': 'req-96deda92-9ffb-4134-9427-e62de8117ea0'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:04.050355 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-96deda92-9ffb-4134-9427-e62de8117ea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd3dc6535b2e4f9aac0568fc20be2739', 'request_id': 'req-96deda92-9ffb-4134-9427-e62de8117ea0'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:04.075360 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:28:04.088755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-96deda92-9ffb-4134-9427-e62de8117ea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd3dc6535b2e4f9aac0568fc20be2739', 'request_id': 'req-96deda92-9ffb-4134-9427-e62de8117ea0'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:04.146400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-96deda92-9ffb-4134-9427-e62de8117ea0 admin admin] The token cache is being invalidate because project bd3dc6535b2e4f9aac0568fc20be2739 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:28:04.147282 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-96deda92-9ffb-4134-9427-e62de8117ea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:04.150624 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-96deda92-9ffb-4134-9427-e62de8117ea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:04.161201 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1110/2196] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:28:03 2022] DELETE /identity/v3/projects/bd3dc6535b2e4f9aac0568fc20be2739 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:28:04.167867 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96deda92-9ffb-4134-9427-e62de8117ea0 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:04.177700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-96deda92-9ffb-4134-9427-e62de8117ea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:04.184829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1087/2197] 10.222.0.88 () {58 vars in 1009 bytes} [Tue Aug 23 22:28:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:28:04.215251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa7850-104d-4be6-9792-73807fe3d6fc None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:04.220054 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-87aa7850-104d-4be6-9792-73807fe3d6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:04.490527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96deda92-9ffb-4134-9427-e62de8117ea0 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:04.520381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ce2dc-93b6-483e-8e5e-525844a183df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:04.522233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ce2dc-93b6-483e-8e5e-525844a183df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:04.525351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ce2dc-93b6-483e-8e5e-525844a183df admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:04.526567 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ce2dc-93b6-483e-8e5e-525844a183df admin admin] PATH_INFO: `/v3/users/9a8d693a13e44f2787612dd1499cea0a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:04.529557 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ce2dc-93b6-483e-8e5e-525844a183df admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a8d693a13e44f2787612dd1499cea0a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:04.535101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ce2dc-93b6-483e-8e5e-525844a183df admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:04.599219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa7850-104d-4be6-9792-73807fe3d6fc None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:04.652613 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:28:04.713968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-430ce2dc-93b6-483e-8e5e-525844a183df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a8d693a13e44f2787612dd1499cea0a'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:04.776538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-430ce2dc-93b6-483e-8e5e-525844a183df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a8d693a13e44f2787612dd1499cea0a'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:04.818403 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-430ce2dc-93b6-483e-8e5e-525844a183df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a8d693a13e44f2787612dd1499cea0a'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:04.819834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-430ce2dc-93b6-483e-8e5e-525844a183df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a8d693a13e44f2787612dd1499cea0a'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:04.841608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-430ce2dc-93b6-483e-8e5e-525844a183df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a8d693a13e44f2787612dd1499cea0a'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:04.870175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1111/2198] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:28:04 2022] DELETE /identity/v3/users/9a8d693a13e44f2787612dd1499cea0a => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:28:04.927618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962e8d3-b20e-4fcd-aa9c-c2b5a57775f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:04.928495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e8d3-b20e-4fcd-aa9c-c2b5a57775f2 service swift] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:04.930689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e8d3-b20e-4fcd-aa9c-c2b5a57775f2 service swift] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:04.931160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e8d3-b20e-4fcd-aa9c-c2b5a57775f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:04.935505 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d962e8d3-b20e-4fcd-aa9c-c2b5a57775f2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:05.134628 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962e8d3-b20e-4fcd-aa9c-c2b5a57775f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:05.150361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962e8d3-b20e-4fcd-aa9c-c2b5a57775f2 service swift] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:05.194561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1088/2199] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:28:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 980 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:28:08.386826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ce2dc-93b6-483e-8e5e-525844a183df admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:08.389690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-430ce2dc-93b6-483e-8e5e-525844a183df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:08.752411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ce2dc-93b6-483e-8e5e-525844a183df admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:08.810935 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907527ff-fd8d-43c7-a1bd-0e61da6120ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:08.814946 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907527ff-fd8d-43c7-a1bd-0e61da6120ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:08.820508 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907527ff-fd8d-43c7-a1bd-0e61da6120ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:08.821789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907527ff-fd8d-43c7-a1bd-0e61da6120ec admin admin] PATH_INFO: `/v3/projects/a805f131a3934c4bad598249d712af81` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:08.830711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907527ff-fd8d-43c7-a1bd-0e61da6120ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=a805f131a3934c4bad598249d712af81)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:08.842763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907527ff-fd8d-43c7-a1bd-0e61da6120ec admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:08.904578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:28:09.080174 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-907527ff-fd8d-43c7-a1bd-0e61da6120ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a805f131a3934c4bad598249d712af81', 'request_id': 'req-907527ff-fd8d-43c7-a1bd-0e61da6120ec'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:09.114272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-907527ff-fd8d-43c7-a1bd-0e61da6120ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a805f131a3934c4bad598249d712af81', 'request_id': 'req-907527ff-fd8d-43c7-a1bd-0e61da6120ec'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:09.149273 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-907527ff-fd8d-43c7-a1bd-0e61da6120ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a805f131a3934c4bad598249d712af81', 'request_id': 'req-907527ff-fd8d-43c7-a1bd-0e61da6120ec'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:09.201347 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-907527ff-fd8d-43c7-a1bd-0e61da6120ec admin admin] The token cache is being invalidate because project a805f131a3934c4bad598249d712af81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:28:09.202002 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-907527ff-fd8d-43c7-a1bd-0e61da6120ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:09.203634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-907527ff-fd8d-43c7-a1bd-0e61da6120ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:09.213535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1112/2200] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:28:08 2022] DELETE /identity/v3/projects/a805f131a3934c4bad598249d712af81 => generated 0 bytes in 822 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:28:09.219218 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e7ce5-f753-44f6-b31b-10d59c4da4d6 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:09.221523 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e7ce5-f753-44f6-b31b-10d59c4da4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:09.222700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e7ce5-f753-44f6-b31b-10d59c4da4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:09.331033 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-e79e7ce5-f753-44f6-b31b-10d59c4da4d6 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:28:09.343255 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e79e7ce5-f753-44f6-b31b-10d59c4da4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:09.628741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1089/2201] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:28:09.642101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819afe6a-5bed-4182-9e7e-9ee2d07950d9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:09.643085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819afe6a-5bed-4182-9e7e-9ee2d07950d9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:09.643085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819afe6a-5bed-4182-9e7e-9ee2d07950d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:09.692854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-819afe6a-5bed-4182-9e7e-9ee2d07950d9 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:28:09.695434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-819afe6a-5bed-4182-9e7e-9ee2d07950d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:09.723939 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1113/2202] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:28:09.725458 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e7ce5-f753-44f6-b31b-10d59c4da4d6 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:09.796334 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e7ce5-f753-44f6-b31b-10d59c4da4d6 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:09.864830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e683fe-4795-4c98-848c-9d2b651703bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:09.867222 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e683fe-4795-4c98-848c-9d2b651703bb admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:09.867222 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e683fe-4795-4c98-848c-9d2b651703bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:09.868673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e683fe-4795-4c98-848c-9d2b651703bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:09.870208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e683fe-4795-4c98-848c-9d2b651703bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:28:09.870741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e683fe-4795-4c98-848c-9d2b651703bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:09.886861 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e683fe-4795-4c98-848c-9d2b651703bb admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:09.937191 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1090/2203] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:28:09.943403 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819afe6a-5bed-4182-9e7e-9ee2d07950d9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:09.984029 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819afe6a-5bed-4182-9e7e-9ee2d07950d9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:10.016341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bbce4-42b7-4fcd-a681-457aff890273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:10.018137 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bbce4-42b7-4fcd-a681-457aff890273 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:10.018821 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bbce4-42b7-4fcd-a681-457aff890273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:10.019520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bbce4-42b7-4fcd-a681-457aff890273 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:10.020794 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bbce4-42b7-4fcd-a681-457aff890273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:10.041364 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bbce4-42b7-4fcd-a681-457aff890273 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:10.171032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1114/2204] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:28:09 2022] POST /identity/v3/projects => generated 373 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:28:10.172477 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e683fe-4795-4c98-848c-9d2b651703bb admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:10.219985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e683fe-4795-4c98-848c-9d2b651703bb admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:10.396399 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:28:10.445344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84f82c-c4ff-46d8-bade-0fcb1a756333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:10.450507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84f82c-c4ff-46d8-bade-0fcb1a756333 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:10.451127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84f82c-c4ff-46d8-bade-0fcb1a756333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:10.451378 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84f82c-c4ff-46d8-bade-0fcb1a756333 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:10.452817 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84f82c-c4ff-46d8-bade-0fcb1a756333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:10.464006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84f82c-c4ff-46d8-bade-0fcb1a756333 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:10.574023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1091/2205] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:28:10 2022] POST /identity/v3/users => generated 287 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:28:10.582063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bbce4-42b7-4fcd-a681-457aff890273 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:10.625377 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bbce4-42b7-4fcd-a681-457aff890273 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:10.644251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec895ae-a17a-4d99-8cb6-df4d1ef073db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:10.650396 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec895ae-a17a-4d99-8cb6-df4d1ef073db admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:10.661035 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec895ae-a17a-4d99-8cb6-df4d1ef073db admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:10.662271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec895ae-a17a-4d99-8cb6-df4d1ef073db admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:10.665393 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec895ae-a17a-4d99-8cb6-df4d1ef073db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:10.669607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec895ae-a17a-4d99-8cb6-df4d1ef073db admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:10.744814 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1115/2206] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:28:10.746364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84f82c-c4ff-46d8-bade-0fcb1a756333 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:10.810509 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84f82c-c4ff-46d8-bade-0fcb1a756333 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:10.843157 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d64a33-e03b-4d72-9a1b-bb1bc0d835b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:10.844256 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d64a33-e03b-4d72-9a1b-bb1bc0d835b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:10.844829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d64a33-e03b-4d72-9a1b-bb1bc0d835b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:10.845366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d64a33-e03b-4d72-9a1b-bb1bc0d835b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:10.851225 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d64a33-e03b-4d72-9a1b-bb1bc0d835b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:10.860015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d64a33-e03b-4d72-9a1b-bb1bc0d835b0 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:10.920146 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1092/2207] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:28:10.925826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec895ae-a17a-4d99-8cb6-df4d1ef073db admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:10.969081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec895ae-a17a-4d99-8cb6-df4d1ef073db admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:10.993412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cccc47-ded2-4e10-969c-ffaf93826cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:10.994908 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cccc47-ded2-4e10-969c-ffaf93826cea admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:10.995833 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cccc47-ded2-4e10-969c-ffaf93826cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:10.996482 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cccc47-ded2-4e10-969c-ffaf93826cea admin admin] PATH_INFO: `/v3/projects/ad8e9c90c2644cedb1831c4efeb0db8e/users/d623f44e8ae84336a27a3550dbd468fd/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:11.073246 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cccc47-ded2-4e10-969c-ffaf93826cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8e9c90c2644cedb1831c4efeb0db8e, user_id=d623f44e8ae84336a27a3550dbd468fd, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:11.082410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cccc47-ded2-4e10-969c-ffaf93826cea admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:11.177738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1116/2208] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:28:10 2022] PUT /identity/v3/projects/ad8e9c90c2644cedb1831c4efeb0db8e/users/d623f44e8ae84336a27a3550dbd468fd/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:28:11.190575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48241878-9792-4fe0-8332-d51348f29921 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:11.192068 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48241878-9792-4fe0-8332-d51348f29921 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:11.193144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48241878-9792-4fe0-8332-d51348f29921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:11.333274 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-48241878-9792-4fe0-8332-d51348f29921 None None] MFA Rules not processed for user `d623f44e8ae84336a27a3550dbd468fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:28:11.347575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-48241878-9792-4fe0-8332-d51348f29921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:11.589406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1093/2209] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:28:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:28:11.608473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cccc47-ded2-4e10-969c-ffaf93826cea admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:11.612692 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-71cccc47-ded2-4e10-969c-ffaf93826cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:11.809495 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cccc47-ded2-4e10-969c-ffaf93826cea admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:11.816711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852121a0-bed7-4e00-b249-a23564c938c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:11.817795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852121a0-bed7-4e00-b249-a23564c938c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:11.818280 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852121a0-bed7-4e00-b249-a23564c938c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:11.818728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852121a0-bed7-4e00-b249-a23564c938c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:11.837884 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852121a0-bed7-4e00-b249-a23564c938c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:11.841426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852121a0-bed7-4e00-b249-a23564c938c9 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:11.875477 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1117/2210] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:28:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:28:11.918718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48241878-9792-4fe0-8332-d51348f29921 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:12.142456 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48241878-9792-4fe0-8332-d51348f29921 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:12.156053 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1d12f-c692-4ef4-82f6-ca5134116349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:12.157180 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1d12f-c692-4ef4-82f6-ca5134116349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:12.157708 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1d12f-c692-4ef4-82f6-ca5134116349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:12.158405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1d12f-c692-4ef4-82f6-ca5134116349 admin admin] PATH_INFO: `/v3/users/d623f44e8ae84336a27a3550dbd468fd` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:12.160568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1d12f-c692-4ef4-82f6-ca5134116349 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d623f44e8ae84336a27a3550dbd468fd)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:12.166504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1d12f-c692-4ef4-82f6-ca5134116349 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:12.321444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-06d1d12f-c692-4ef4-82f6-ca5134116349 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd623f44e8ae84336a27a3550dbd468fd'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:12.345554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-06d1d12f-c692-4ef4-82f6-ca5134116349 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd623f44e8ae84336a27a3550dbd468fd'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:12.379100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-06d1d12f-c692-4ef4-82f6-ca5134116349 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd623f44e8ae84336a27a3550dbd468fd'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:12.383517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:28:12.455957 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-06d1d12f-c692-4ef4-82f6-ca5134116349 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd623f44e8ae84336a27a3550dbd468fd'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:12.491739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-06d1d12f-c692-4ef4-82f6-ca5134116349 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd623f44e8ae84336a27a3550dbd468fd'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:12.508676 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1094/2211] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:28:11 2022] DELETE /identity/v3/users/d623f44e8ae84336a27a3550dbd468fd => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:28:12.518125 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc773a2b-f762-42fe-a40e-6534b9c7bd7f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:12.518775 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc773a2b-f762-42fe-a40e-6534b9c7bd7f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:12.519346 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc773a2b-f762-42fe-a40e-6534b9c7bd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:12.620568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fc773a2b-f762-42fe-a40e-6534b9c7bd7f None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:28:12.623251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fc773a2b-f762-42fe-a40e-6534b9c7bd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:12.842453 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:28:13.031160 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1118/2212] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 517 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:28:13.085788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1d12f-c692-4ef4-82f6-ca5134116349 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:13.090542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-06d1d12f-c692-4ef4-82f6-ca5134116349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:13.436899 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1d12f-c692-4ef4-82f6-ca5134116349 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:13.463065 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8eee6-21c5-4939-a90b-e4a41bb4e566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:13.464301 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8eee6-21c5-4939-a90b-e4a41bb4e566 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:13.465113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8eee6-21c5-4939-a90b-e4a41bb4e566 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:13.465803 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8eee6-21c5-4939-a90b-e4a41bb4e566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:13.525183 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8eee6-21c5-4939-a90b-e4a41bb4e566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:13.530377 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8eee6-21c5-4939-a90b-e4a41bb4e566 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:13.565609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1095/2213] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:28:15.666100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc773a2b-f762-42fe-a40e-6534b9c7bd7f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:15.668549 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fc773a2b-f762-42fe-a40e-6534b9c7bd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:15.754815 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc773a2b-f762-42fe-a40e-6534b9c7bd7f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:15.806606 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79114f48-0a57-457b-b58d-f07d67d83681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:15.814473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79114f48-0a57-457b-b58d-f07d67d83681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:15.820831 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79114f48-0a57-457b-b58d-f07d67d83681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:15.821629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79114f48-0a57-457b-b58d-f07d67d83681 admin admin] PATH_INFO: `/v3/projects/ad8e9c90c2644cedb1831c4efeb0db8e` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:15.827044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79114f48-0a57-457b-b58d-f07d67d83681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad8e9c90c2644cedb1831c4efeb0db8e)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:15.833964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79114f48-0a57-457b-b58d-f07d67d83681 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:16.023650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-79114f48-0a57-457b-b58d-f07d67d83681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad8e9c90c2644cedb1831c4efeb0db8e', 'request_id': 'req-79114f48-0a57-457b-b58d-f07d67d83681'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:16.061728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-79114f48-0a57-457b-b58d-f07d67d83681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad8e9c90c2644cedb1831c4efeb0db8e', 'request_id': 'req-79114f48-0a57-457b-b58d-f07d67d83681'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:16.090317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-79114f48-0a57-457b-b58d-f07d67d83681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad8e9c90c2644cedb1831c4efeb0db8e', 'request_id': 'req-79114f48-0a57-457b-b58d-f07d67d83681'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:16.138606 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-79114f48-0a57-457b-b58d-f07d67d83681 admin admin] The token cache is being invalidate because project ad8e9c90c2644cedb1831c4efeb0db8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:28:16.139101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-79114f48-0a57-457b-b58d-f07d67d83681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:16.140344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-79114f48-0a57-457b-b58d-f07d67d83681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:28:16.143861 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1119/2214] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:28:15 2022] DELETE /identity/v3/projects/ad8e9c90c2644cedb1831c4efeb0db8e => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:28:16.151559 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacb0f4-8760-4c7d-af41-dbf82dc455fa None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:16.152601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacb0f4-8760-4c7d-af41-dbf82dc455fa None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:16.153603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacb0f4-8760-4c7d-af41-dbf82dc455fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:16.214881 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-3cacb0f4-8760-4c7d-af41-dbf82dc455fa None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:28:16.218687 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3cacb0f4-8760-4c7d-af41-dbf82dc455fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:16.525420 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1096/2215] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:28:16.534825 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c8ca8-3051-4e8d-9ced-1cb3c977c652 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:16.535504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c8ca8-3051-4e8d-9ced-1cb3c977c652 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:16.536032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c8ca8-3051-4e8d-9ced-1cb3c977c652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:16.608636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-966c8ca8-3051-4e8d-9ced-1cb3c977c652 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:28:16.612040 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-966c8ca8-3051-4e8d-9ced-1cb3c977c652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:16.640320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1120/2216] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:28:16.648020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacb0f4-8760-4c7d-af41-dbf82dc455fa None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:16.682844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacb0f4-8760-4c7d-af41-dbf82dc455fa None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:16.702042 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a718a-c7ac-48f0-a115-7999a4b5e17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:16.704366 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a718a-c7ac-48f0-a115-7999a4b5e17f admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:16.708367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a718a-c7ac-48f0-a115-7999a4b5e17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:16.710234 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a718a-c7ac-48f0-a115-7999a4b5e17f admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:16.713722 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a718a-c7ac-48f0-a115-7999a4b5e17f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:28:16.714826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a718a-c7ac-48f0-a115-7999a4b5e17f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:16.731179 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a718a-c7ac-48f0-a115-7999a4b5e17f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:16.781712 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1097/2217] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:28:16.791305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c8ca8-3051-4e8d-9ced-1cb3c977c652 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:16.838615 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c8ca8-3051-4e8d-9ced-1cb3c977c652 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:16.852649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305097b-0dbe-43ab-aaef-537604f2a731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:16.854624 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305097b-0dbe-43ab-aaef-537604f2a731 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:16.855396 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305097b-0dbe-43ab-aaef-537604f2a731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:16.856093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305097b-0dbe-43ab-aaef-537604f2a731 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:16.858117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305097b-0dbe-43ab-aaef-537604f2a731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:16.868663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305097b-0dbe-43ab-aaef-537604f2a731 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:16.958725 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1121/2218] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:28:16 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:28:16.970856 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a718a-c7ac-48f0-a115-7999a4b5e17f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:17.000589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a718a-c7ac-48f0-a115-7999a4b5e17f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:17.314453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d3991-efe3-4096-a2c1-0b681a27c3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:17.315896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d3991-efe3-4096-a2c1-0b681a27c3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:17.316497 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d3991-efe3-4096-a2c1-0b681a27c3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:17.317001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d3991-efe3-4096-a2c1-0b681a27c3a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:17.320945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d3991-efe3-4096-a2c1-0b681a27c3a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:17.329746 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d3991-efe3-4096-a2c1-0b681a27c3a4 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:17.377437 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:28:17.473145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1098/2219] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:28:16 2022] POST /identity/v3/users => generated 290 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:28:17.473878 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305097b-0dbe-43ab-aaef-537604f2a731 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:17.511377 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305097b-0dbe-43ab-aaef-537604f2a731 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:17.557750 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942a5da-258c-4eef-ab86-a16386ce2dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:17.558369 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942a5da-258c-4eef-ab86-a16386ce2dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:17.564427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942a5da-258c-4eef-ab86-a16386ce2dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:17.570116 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942a5da-258c-4eef-ab86-a16386ce2dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:17.571585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942a5da-258c-4eef-ab86-a16386ce2dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:17.576999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942a5da-258c-4eef-ab86-a16386ce2dd3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:17.637956 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1122/2220] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:28:17.646105 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d3991-efe3-4096-a2c1-0b681a27c3a4 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:17.698521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d3991-efe3-4096-a2c1-0b681a27c3a4 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:17.725648 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc1b9b-c3c2-49b2-94b1-33b82bde5ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:17.727206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc1b9b-c3c2-49b2-94b1-33b82bde5ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:17.727698 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc1b9b-c3c2-49b2-94b1-33b82bde5ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:17.728177 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc1b9b-c3c2-49b2-94b1-33b82bde5ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:17.745104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc1b9b-c3c2-49b2-94b1-33b82bde5ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:17.745104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc1b9b-c3c2-49b2-94b1-33b82bde5ba5 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:17.830075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1099/2221] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:28:17.832128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942a5da-258c-4eef-ab86-a16386ce2dd3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:17.874864 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942a5da-258c-4eef-ab86-a16386ce2dd3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:17.892528 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea879e19-b5cb-4e71-acf2-162d831fd085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:17.894123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea879e19-b5cb-4e71-acf2-162d831fd085 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:17.895048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea879e19-b5cb-4e71-acf2-162d831fd085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:17.895689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea879e19-b5cb-4e71-acf2-162d831fd085 admin admin] PATH_INFO: `/v3/projects/73aca04a5685447fb698a46da1b0b8ff/users/5918150a21314cf2b28bcee3072457ae/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:17.963114 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea879e19-b5cb-4e71-acf2-162d831fd085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73aca04a5685447fb698a46da1b0b8ff, user_id=5918150a21314cf2b28bcee3072457ae, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:17.979077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea879e19-b5cb-4e71-acf2-162d831fd085 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:18.028354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1123/2222] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:28:17 2022] PUT /identity/v3/projects/73aca04a5685447fb698a46da1b0b8ff/users/5918150a21314cf2b28bcee3072457ae/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:28:18.040897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41975b-8917-4c69-a8b8-9499108ffc44 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:18.041842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41975b-8917-4c69-a8b8-9499108ffc44 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:18.042424 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41975b-8917-4c69-a8b8-9499108ffc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:18.139554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-4d41975b-8917-4c69-a8b8-9499108ffc44 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:28:18.143659 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4d41975b-8917-4c69-a8b8-9499108ffc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:18.412085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1100/2223] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:28:18.434626 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea879e19-b5cb-4e71-acf2-162d831fd085 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:18.438508 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ea879e19-b5cb-4e71-acf2-162d831fd085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:18.642918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea879e19-b5cb-4e71-acf2-162d831fd085 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:18.665290 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23fd0b-5d7a-4cff-82b6-aa937c2978e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:18.670613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23fd0b-5d7a-4cff-82b6-aa937c2978e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:18.671528 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23fd0b-5d7a-4cff-82b6-aa937c2978e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:18.672504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23fd0b-5d7a-4cff-82b6-aa937c2978e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:18.698246 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23fd0b-5d7a-4cff-82b6-aa937c2978e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:18.701627 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23fd0b-5d7a-4cff-82b6-aa937c2978e0 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:18.728604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1124/2224] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:28:20.301001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3ca5e-96c9-4551-b9cb-bffd01fab50b None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:20.301715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3ca5e-96c9-4551-b9cb-bffd01fab50b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:20.302151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3ca5e-96c9-4551-b9cb-bffd01fab50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:20.353765 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-1be3ca5e-96c9-4551-b9cb-bffd01fab50b None None] MFA Rules not processed for user `5918150a21314cf2b28bcee3072457ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:28:20.355219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-1be3ca5e-96c9-4551-b9cb-bffd01fab50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:20.477446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1101/2225] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:28:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:28:20.499609 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23fd0b-5d7a-4cff-82b6-aa937c2978e0 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:20.503275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ba23fd0b-5d7a-4cff-82b6-aa937c2978e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:20.511949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:28:20.637092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23fd0b-5d7a-4cff-82b6-aa937c2978e0 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:20.645305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5d79a-67ca-4b9a-8ead-f83127475d15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:20.647732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5d79a-67ca-4b9a-8ead-f83127475d15 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:20.648384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5d79a-67ca-4b9a-8ead-f83127475d15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:20.648842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5d79a-67ca-4b9a-8ead-f83127475d15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:20.668380 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5d79a-67ca-4b9a-8ead-f83127475d15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:20.671304 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5d79a-67ca-4b9a-8ead-f83127475d15 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:20.697659 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1125/2226] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:28:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:28:22.484661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b431f-668f-4290-b88c-2ea422f65773 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:22.485569 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b431f-668f-4290-b88c-2ea422f65773 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:22.485717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b431f-668f-4290-b88c-2ea422f65773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:22.512631 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5c9b431f-668f-4290-b88c-2ea422f65773 None None] MFA Rules not processed for user `5918150a21314cf2b28bcee3072457ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:28:22.515018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5c9b431f-668f-4290-b88c-2ea422f65773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:22.528662 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1102/2227] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:28:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:28:30.683480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=25 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:28:32.503343 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:INSERT=1,keystone:SELECT=69 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:28:35.234672 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f698c-484d-45c8-bec0-cf2aafb5fe60 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:35.235526 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f698c-484d-45c8-bec0-cf2aafb5fe60 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:35.235526 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f698c-484d-45c8-bec0-cf2aafb5fe60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:35.260079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-369f698c-484d-45c8-bec0-cf2aafb5fe60 None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:28:35.261545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-369f698c-484d-45c8-bec0-cf2aafb5fe60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:35.349890 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1126/2228] 10.222.0.88 () {58 vars in 967 bytes} [Tue Aug 23 22:28:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:28:35.385461 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b431f-668f-4290-b88c-2ea422f65773 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:35.388252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5c9b431f-668f-4290-b88c-2ea422f65773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:35.515276 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b431f-668f-4290-b88c-2ea422f65773 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:35.526333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9da8e7-458c-4fba-ad2b-e52ed69beb40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:35.527108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9da8e7-458c-4fba-ad2b-e52ed69beb40 service swift] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:35.527499 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9da8e7-458c-4fba-ad2b-e52ed69beb40 service swift] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:35.527961 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9da8e7-458c-4fba-ad2b-e52ed69beb40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:35.554295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9da8e7-458c-4fba-ad2b-e52ed69beb40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:35.558534 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9da8e7-458c-4fba-ad2b-e52ed69beb40 service swift] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:35.579220 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1103/2229] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:28:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:28:45.336668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=25 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:28:45.571546 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=32 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:29:19.589934 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f698c-484d-45c8-bec0-cf2aafb5fe60 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:29:19.591761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-369f698c-484d-45c8-bec0-cf2aafb5fe60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:19.615284 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f698c-484d-45c8-bec0-cf2aafb5fe60 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:29:19.622428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c8f94-c857-457e-b2d8-6acf90c87657 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:29:19.623519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c8f94-c857-457e-b2d8-6acf90c87657 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:19.624898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c8f94-c857-457e-b2d8-6acf90c87657 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:19.625446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c8f94-c857-457e-b2d8-6acf90c87657 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:19.628774 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ea8c8f94-c857-457e-b2d8-6acf90c87657 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:19.754629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c8f94-c857-457e-b2d8-6acf90c87657 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:29:19.756609 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c8f94-c857-457e-b2d8-6acf90c87657 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:29:19.773737 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1127/2230] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:29:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:29:20.940258 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9da8e7-458c-4fba-ad2b-e52ed69beb40 service swift] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:29:20.953170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9da8e7-458c-4fba-ad2b-e52ed69beb40 service swift] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:29:20.958914 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b388c80-e11b-463f-8f98-0fcc9480b787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:29:20.959582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b388c80-e11b-463f-8f98-0fcc9480b787 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:20.959870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b388c80-e11b-463f-8f98-0fcc9480b787 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:20.960187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b388c80-e11b-463f-8f98-0fcc9480b787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:20.962355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-7b388c80-e11b-463f-8f98-0fcc9480b787 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:20.984484 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b388c80-e11b-463f-8f98-0fcc9480b787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:29:20.986464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b388c80-e11b-463f-8f98-0fcc9480b787 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:29:21.002984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1104/2231] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:29:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:29:28.845665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f22ef-7942-46ea-b979-9e06b8832dad None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:28.846405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f22ef-7942-46ea-b979-9e06b8832dad None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:28.846405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f22ef-7942-46ea-b979-9e06b8832dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:28.875415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c06f22ef-7942-46ea-b979-9e06b8832dad None None] MFA Rules not processed for user `1db1a9a77be6452991ec4f99af814e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:29:28.876916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c06f22ef-7942-46ea-b979-9e06b8832dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:28.984763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1128/2232] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:29:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:29:29.325580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba115fd-8899-42c9-8de6-665fdff3ebf0 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:29.326495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba115fd-8899-42c9-8de6-665fdff3ebf0 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:29.326495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba115fd-8899-42c9-8de6-665fdff3ebf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:29.373567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5ba115fd-8899-42c9-8de6-665fdff3ebf0 None None] MFA Rules not processed for user `78b05a538ce0466ea8b3e27a0a6fc162`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:29:29.376207 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5ba115fd-8899-42c9-8de6-665fdff3ebf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:29.495414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1105/2233] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:29:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:29:38.974356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=59 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:29:39.480499 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=31 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:29:41.828747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f22ef-7942-46ea-b979-9e06b8832dad None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:29:41.830317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c06f22ef-7942-46ea-b979-9e06b8832dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:41.863031 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f22ef-7942-46ea-b979-9e06b8832dad None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:29:41.875061 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62b135-c639-4b72-86ac-2954fc38cedd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:29:41.876593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62b135-c639-4b72-86ac-2954fc38cedd service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:41.877217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62b135-c639-4b72-86ac-2954fc38cedd service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:41.877789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62b135-c639-4b72-86ac-2954fc38cedd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:41.882114 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3e62b135-c639-4b72-86ac-2954fc38cedd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:41.904336 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62b135-c639-4b72-86ac-2954fc38cedd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:29:41.908020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62b135-c639-4b72-86ac-2954fc38cedd service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:29:41.946742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1129/2234] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:29:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:29:43.383482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba115fd-8899-42c9-8de6-665fdff3ebf0 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:29:43.385519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5ba115fd-8899-42c9-8de6-665fdff3ebf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:43.404458 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba115fd-8899-42c9-8de6-665fdff3ebf0 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:29:43.411277 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edfcf1-7744-4fdd-adfd-a9c8921dbfa9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:29:43.412085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edfcf1-7744-4fdd-adfd-a9c8921dbfa9 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:43.412464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edfcf1-7744-4fdd-adfd-a9c8921dbfa9 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:43.412847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edfcf1-7744-4fdd-adfd-a9c8921dbfa9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:43.415325 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-34edfcf1-7744-4fdd-adfd-a9c8921dbfa9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:43.539779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34edfcf1-7744-4fdd-adfd-a9c8921dbfa9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:29:43.541680 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34edfcf1-7744-4fdd-adfd-a9c8921dbfa9 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:29:43.557164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1106/2235] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:29:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:29:46.291825 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62b135-c639-4b72-86ac-2954fc38cedd service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:29:46.304308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62b135-c639-4b72-86ac-2954fc38cedd service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:29:46.308804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17eae11-00c6-42ac-a750-f904c6679041 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:29:46.309377 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17eae11-00c6-42ac-a750-f904c6679041 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:46.309806 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17eae11-00c6-42ac-a750-f904c6679041 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:46.310083 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17eae11-00c6-42ac-a750-f904c6679041 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:46.312072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c17eae11-00c6-42ac-a750-f904c6679041 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:46.382980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38e36a-3805-404d-ba7f-5912b44ad983 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:46.383904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38e36a-3805-404d-ba7f-5912b44ad983 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:46.383904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38e36a-3805-404d-ba7f-5912b44ad983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:46.392360 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17eae11-00c6-42ac-a750-f904c6679041 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:29:46.395930 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17eae11-00c6-42ac-a750-f904c6679041 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:29:46.405777 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-2f38e36a-3805-404d-ba7f-5912b44ad983 None None] MFA Rules not processed for user `5918150a21314cf2b28bcee3072457ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:29:46.407051 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-2f38e36a-3805-404d-ba7f-5912b44ad983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:46.413360 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1130/2236] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:29:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:29:46.415873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1107/2237] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:29:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:29:56.402355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=31 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:29:56.403337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=34 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:01.501985 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05431036-5032-489b-80b6-d8b931df55b0 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:01.501985 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05431036-5032-489b-80b6-d8b931df55b0 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:01.501985 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05431036-5032-489b-80b6-d8b931df55b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:01.547332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-05431036-5032-489b-80b6-d8b931df55b0 None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:01.549127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-05431036-5032-489b-80b6-d8b931df55b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:01.565073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1131/2238] 10.222.0.88 () {58 vars in 967 bytes} [Tue Aug 23 22:30:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:01.635498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38e36a-3805-404d-ba7f-5912b44ad983 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:01.651967 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38e36a-3805-404d-ba7f-5912b44ad983 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:01.658029 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda4b16-7286-4ddf-b14f-5e2fcaac6ed5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:01.658679 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda4b16-7286-4ddf-b14f-5e2fcaac6ed5 service swift] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:01.658961 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda4b16-7286-4ddf-b14f-5e2fcaac6ed5 service swift] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:01.659233 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda4b16-7286-4ddf-b14f-5e2fcaac6ed5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:01.672639 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda4b16-7286-4ddf-b14f-5e2fcaac6ed5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:01.674528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda4b16-7286-4ddf-b14f-5e2fcaac6ed5 service swift] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:01.692563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1108/2239] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:30:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 22:30:03.189176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898c4e6-0c8e-4d13-b2ef-3b3d29378b02 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:03.189872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898c4e6-0c8e-4d13-b2ef-3b3d29378b02 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:03.189872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898c4e6-0c8e-4d13-b2ef-3b3d29378b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:03.212712 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d898c4e6-0c8e-4d13-b2ef-3b3d29378b02 None None] MFA Rules not processed for user `e720fd25d54b4dc6b3387ea762d7e4ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:03.214096 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d898c4e6-0c8e-4d13-b2ef-3b3d29378b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:03.221491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1132/2240] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:30:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:04.311363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda4b16-7286-4ddf-b14f-5e2fcaac6ed5 service swift] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:04.343911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda4b16-7286-4ddf-b14f-5e2fcaac6ed5 service swift] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:04.352712 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529cdd9-aeec-4a89-9e27-f5a1f54e43da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:04.354106 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529cdd9-aeec-4a89-9e27-f5a1f54e43da service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:04.354953 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529cdd9-aeec-4a89-9e27-f5a1f54e43da service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:04.355397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529cdd9-aeec-4a89-9e27-f5a1f54e43da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:04.358656 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f529cdd9-aeec-4a89-9e27-f5a1f54e43da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:04.371500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529cdd9-aeec-4a89-9e27-f5a1f54e43da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:04.373683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529cdd9-aeec-4a89-9e27-f5a1f54e43da service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:04.386465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1109/2241] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:30:05.090374 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898c4e6-0c8e-4d13-b2ef-3b3d29378b02 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:05.093887 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d898c4e6-0c8e-4d13-b2ef-3b3d29378b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:05.232071 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898c4e6-0c8e-4d13-b2ef-3b3d29378b02 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:05.241190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664cd624-d0ae-49cb-a35a-9c01113b81df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e642dd7a07ca4afaa40c2769299d8bad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:05.242241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664cd624-d0ae-49cb-a35a-9c01113b81df service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:05.242779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664cd624-d0ae-49cb-a35a-9c01113b81df service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:05.243286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664cd624-d0ae-49cb-a35a-9c01113b81df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:05.245743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-664cd624-d0ae-49cb-a35a-9c01113b81df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:05.272528 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664cd624-d0ae-49cb-a35a-9c01113b81df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:05.274508 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664cd624-d0ae-49cb-a35a-9c01113b81df service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:05.294823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1133/2242] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:30:05.310586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872f885-cdd2-4589-a5e5-247badb79203 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:05.310865 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872f885-cdd2-4589-a5e5-247badb79203 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:05.311129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872f885-cdd2-4589-a5e5-247badb79203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:05.339432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-6872f885-cdd2-4589-a5e5-247badb79203 None None] MFA Rules not processed for user `c06cd43d533f4621b0a69dbb0ef22dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:05.342821 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6872f885-cdd2-4589-a5e5-247badb79203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:05.357413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1110/2243] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:30:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:10.843010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664cd624-d0ae-49cb-a35a-9c01113b81df service placement] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:10.863874 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664cd624-d0ae-49cb-a35a-9c01113b81df service placement] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:10.870862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72589092-d1a5-491f-bf78-bc5eab2adae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:10.871994 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72589092-d1a5-491f-bf78-bc5eab2adae2 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:10.872313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72589092-d1a5-491f-bf78-bc5eab2adae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:10.872730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72589092-d1a5-491f-bf78-bc5eab2adae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:10.875687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-72589092-d1a5-491f-bf78-bc5eab2adae2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:10.889201 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72589092-d1a5-491f-bf78-bc5eab2adae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:10.890879 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72589092-d1a5-491f-bf78-bc5eab2adae2 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:10.901929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1134/2244] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:30:15.333590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=15 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:16.478394 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872f885-cdd2-4589-a5e5-247badb79203 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:16.481863 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6872f885-cdd2-4589-a5e5-247badb79203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:16.499653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872f885-cdd2-4589-a5e5-247badb79203 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:16.506372 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5ac40-2c09-4018-bd14-21046e064b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:16.507290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5ac40-2c09-4018-bd14-21046e064b30 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:16.507754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5ac40-2c09-4018-bd14-21046e064b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:16.508248 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5ac40-2c09-4018-bd14-21046e064b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:16.511125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-abc5ac40-2c09-4018-bd14-21046e064b30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:16.531470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5ac40-2c09-4018-bd14-21046e064b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:16.534995 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5ac40-2c09-4018-bd14-21046e064b30 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:16.556794 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1111/2245] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:16 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:30:19.342892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72589092-d1a5-491f-bf78-bc5eab2adae2 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:19.351677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-72589092-d1a5-491f-bf78-bc5eab2adae2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:19.382303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72589092-d1a5-491f-bf78-bc5eab2adae2 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:19.393238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fa09f-9a2e-4df8-bfd5-7ca27094b854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:19.395037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa09f-9a2e-4df8-bfd5-7ca27094b854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:19.395678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa09f-9a2e-4df8-bfd5-7ca27094b854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:19.396320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa09f-9a2e-4df8-bfd5-7ca27094b854 admin admin] PATH_INFO: `/v3/users/eb81364013af425e9956dd09ceb150e6` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:19.398847 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fa09f-9a2e-4df8-bfd5-7ca27094b854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb81364013af425e9956dd09ceb150e6)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:19.403528 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fa09f-9a2e-4df8-bfd5-7ca27094b854 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:19.576419 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1c1fa09f-9a2e-4df8-bfd5-7ca27094b854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb81364013af425e9956dd09ceb150e6'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:19.612104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1c1fa09f-9a2e-4df8-bfd5-7ca27094b854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb81364013af425e9956dd09ceb150e6'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:19.635187 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1c1fa09f-9a2e-4df8-bfd5-7ca27094b854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb81364013af425e9956dd09ceb150e6'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:19.637363 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1c1fa09f-9a2e-4df8-bfd5-7ca27094b854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb81364013af425e9956dd09ceb150e6'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:19.656023 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1c1fa09f-9a2e-4df8-bfd5-7ca27094b854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb81364013af425e9956dd09ceb150e6'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:19.666057 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1135/2246] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:30:19 2022] DELETE /identity/v3/users/eb81364013af425e9956dd09ceb150e6 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:19.975926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5ac40-2c09-4018-bd14-21046e064b30 service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:20.012627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5ac40-2c09-4018-bd14-21046e064b30 service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:20.187201 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275fde7-8ef2-45e5-aa5f-343bbf11690f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:20.188228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275fde7-8ef2-45e5-aa5f-343bbf11690f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:20.188711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275fde7-8ef2-45e5-aa5f-343bbf11690f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:20.189231 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275fde7-8ef2-45e5-aa5f-343bbf11690f admin admin] PATH_INFO: `/v3/projects/2cfc99aa2ea246fb845d429885ff5209` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:20.191813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275fde7-8ef2-45e5-aa5f-343bbf11690f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cfc99aa2ea246fb845d429885ff5209)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:20.194851 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275fde7-8ef2-45e5-aa5f-343bbf11690f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:20.303229 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f275fde7-8ef2-45e5-aa5f-343bbf11690f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cfc99aa2ea246fb845d429885ff5209', 'request_id': 'req-f275fde7-8ef2-45e5-aa5f-343bbf11690f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:20.314849 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f275fde7-8ef2-45e5-aa5f-343bbf11690f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cfc99aa2ea246fb845d429885ff5209', 'request_id': 'req-f275fde7-8ef2-45e5-aa5f-343bbf11690f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:20.325457 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f275fde7-8ef2-45e5-aa5f-343bbf11690f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cfc99aa2ea246fb845d429885ff5209', 'request_id': 'req-f275fde7-8ef2-45e5-aa5f-343bbf11690f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:20.348760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f275fde7-8ef2-45e5-aa5f-343bbf11690f admin admin] The token cache is being invalidate because project 2cfc99aa2ea246fb845d429885ff5209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:30:20.349438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f275fde7-8ef2-45e5-aa5f-343bbf11690f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:20.350962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f275fde7-8ef2-45e5-aa5f-343bbf11690f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:20.354572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1112/2247] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:30:19 2022] DELETE /identity/v3/projects/2cfc99aa2ea246fb845d429885ff5209 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:20.361455 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fa09f-9a2e-4df8-bfd5-7ca27094b854 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:20.365477 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1c1fa09f-9a2e-4df8-bfd5-7ca27094b854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:20.391381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fa09f-9a2e-4df8-bfd5-7ca27094b854 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:20.474894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:20.529592 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b815c-4ed5-4911-8557-f8cbb83e04e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:20.531327 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b815c-4ed5-4911-8557-f8cbb83e04e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:20.532137 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b815c-4ed5-4911-8557-f8cbb83e04e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:20.532590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b815c-4ed5-4911-8557-f8cbb83e04e4 admin admin] PATH_INFO: `/v3/users/5c452a85ac484f89b2f1d6fa8fa55f6e` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:20.539242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b815c-4ed5-4911-8557-f8cbb83e04e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c452a85ac484f89b2f1d6fa8fa55f6e)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:20.542775 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b815c-4ed5-4911-8557-f8cbb83e04e4 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:20.628882 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2c8b815c-4ed5-4911-8557-f8cbb83e04e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c452a85ac484f89b2f1d6fa8fa55f6e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:20.648114 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2c8b815c-4ed5-4911-8557-f8cbb83e04e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c452a85ac484f89b2f1d6fa8fa55f6e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:20.665098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2c8b815c-4ed5-4911-8557-f8cbb83e04e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c452a85ac484f89b2f1d6fa8fa55f6e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:20.666172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2c8b815c-4ed5-4911-8557-f8cbb83e04e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c452a85ac484f89b2f1d6fa8fa55f6e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:20.683848 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2c8b815c-4ed5-4911-8557-f8cbb83e04e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c452a85ac484f89b2f1d6fa8fa55f6e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:20.701833 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1136/2248] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:30:20 2022] DELETE /identity/v3/users/5c452a85ac484f89b2f1d6fa8fa55f6e => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:21.055270 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275fde7-8ef2-45e5-aa5f-343bbf11690f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:21.058833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f275fde7-8ef2-45e5-aa5f-343bbf11690f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:21.103992 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275fde7-8ef2-45e5-aa5f-343bbf11690f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:21.223573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5242e5b-318b-40e4-86a3-2ff3cf7b408f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:21.224678 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5242e5b-318b-40e4-86a3-2ff3cf7b408f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:21.225273 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5242e5b-318b-40e4-86a3-2ff3cf7b408f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:21.225846 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5242e5b-318b-40e4-86a3-2ff3cf7b408f admin admin] PATH_INFO: `/v3/projects/e1969acf59134ccca626d7cb1015a184` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:21.227942 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5242e5b-318b-40e4-86a3-2ff3cf7b408f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1969acf59134ccca626d7cb1015a184)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:21.231807 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5242e5b-318b-40e4-86a3-2ff3cf7b408f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:21.275641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:21.321630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e5242e5b-318b-40e4-86a3-2ff3cf7b408f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1969acf59134ccca626d7cb1015a184', 'request_id': 'req-e5242e5b-318b-40e4-86a3-2ff3cf7b408f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:21.330082 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e5242e5b-318b-40e4-86a3-2ff3cf7b408f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1969acf59134ccca626d7cb1015a184', 'request_id': 'req-e5242e5b-318b-40e4-86a3-2ff3cf7b408f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:21.338421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e5242e5b-318b-40e4-86a3-2ff3cf7b408f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1969acf59134ccca626d7cb1015a184', 'request_id': 'req-e5242e5b-318b-40e4-86a3-2ff3cf7b408f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:21.360135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e5242e5b-318b-40e4-86a3-2ff3cf7b408f admin admin] The token cache is being invalidate because project e1969acf59134ccca626d7cb1015a184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:30:21.360659 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e5242e5b-318b-40e4-86a3-2ff3cf7b408f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:21.361886 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e5242e5b-318b-40e4-86a3-2ff3cf7b408f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:30:21.365337 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1113/2249] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:30:21 2022] DELETE /identity/v3/projects/e1969acf59134ccca626d7cb1015a184 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:21.371483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b904e7a-f918-469d-a922-2ff31a47f334 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:21.372709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b904e7a-f918-469d-a922-2ff31a47f334 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:21.373009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b904e7a-f918-469d-a922-2ff31a47f334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:21.404221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2b904e7a-f918-469d-a922-2ff31a47f334 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:21.405792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2b904e7a-f918-469d-a922-2ff31a47f334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:21.509631 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1137/2250] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:21.524029 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a814706-13e9-47bf-b70d-7b8014d13bcb None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:21.524333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a814706-13e9-47bf-b70d-7b8014d13bcb None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:21.524622 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a814706-13e9-47bf-b70d-7b8014d13bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:21.551871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-0a814706-13e9-47bf-b70d-7b8014d13bcb None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:21.553318 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0a814706-13e9-47bf-b70d-7b8014d13bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:21.576912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1114/2251] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:21.583180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b904e7a-f918-469d-a922-2ff31a47f334 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:21.602560 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b904e7a-f918-469d-a922-2ff31a47f334 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:21.608318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1e33b-e802-4a36-a4f0-e63094860227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:21.609158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1e33b-e802-4a36-a4f0-e63094860227 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:21.609473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1e33b-e802-4a36-a4f0-e63094860227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:21.610580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1e33b-e802-4a36-a4f0-e63094860227 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:21.610770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1e33b-e802-4a36-a4f0-e63094860227 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:30:21.610914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1e33b-e802-4a36-a4f0-e63094860227 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:21.613523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1e33b-e802-4a36-a4f0-e63094860227 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:21.644907 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1138/2252] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:30:21.652292 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a814706-13e9-47bf-b70d-7b8014d13bcb None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:21.675970 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a814706-13e9-47bf-b70d-7b8014d13bcb None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:21.685946 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61e7df-6eae-40ee-9301-d7575f6a24c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:21.687077 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61e7df-6eae-40ee-9301-d7575f6a24c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:21.687586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61e7df-6eae-40ee-9301-d7575f6a24c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:21.688084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61e7df-6eae-40ee-9301-d7575f6a24c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:21.689436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61e7df-6eae-40ee-9301-d7575f6a24c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:21.694079 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61e7df-6eae-40ee-9301-d7575f6a24c7 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:21.748196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1115/2253] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:30:21 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:21.753705 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1e33b-e802-4a36-a4f0-e63094860227 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:21.773762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1e33b-e802-4a36-a4f0-e63094860227 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:21.845849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:21.881562 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256b19a-8aac-4492-a443-c8b26ea2f839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:21.882438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256b19a-8aac-4492-a443-c8b26ea2f839 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:21.882753 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256b19a-8aac-4492-a443-c8b26ea2f839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:21.883154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256b19a-8aac-4492-a443-c8b26ea2f839 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:21.884035 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256b19a-8aac-4492-a443-c8b26ea2f839 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:21.886205 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256b19a-8aac-4492-a443-c8b26ea2f839 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:21.920984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1139/2254] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:30:21 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:21.932209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61e7df-6eae-40ee-9301-d7575f6a24c7 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:21.944609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61e7df-6eae-40ee-9301-d7575f6a24c7 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:21.949831 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb1e45-7251-463a-a0e6-ca127622f5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:21.950463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb1e45-7251-463a-a0e6-ca127622f5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:21.950740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb1e45-7251-463a-a0e6-ca127622f5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:21.951012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb1e45-7251-463a-a0e6-ca127622f5b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:21.951669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb1e45-7251-463a-a0e6-ca127622f5b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:21.953350 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb1e45-7251-463a-a0e6-ca127622f5b1 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:21.964713 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1116/2255] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:30:21.972775 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256b19a-8aac-4492-a443-c8b26ea2f839 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:21.990303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256b19a-8aac-4492-a443-c8b26ea2f839 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:21.998967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478105e-0e1d-4a87-9d92-da3caeff3571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:21.999988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478105e-0e1d-4a87-9d92-da3caeff3571 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:22.000504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478105e-0e1d-4a87-9d92-da3caeff3571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:22.001000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478105e-0e1d-4a87-9d92-da3caeff3571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:22.002304 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478105e-0e1d-4a87-9d92-da3caeff3571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:22.007061 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478105e-0e1d-4a87-9d92-da3caeff3571 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:22.022451 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1140/2256] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:30:22.026943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb1e45-7251-463a-a0e6-ca127622f5b1 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:22.039955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb1e45-7251-463a-a0e6-ca127622f5b1 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:22.045024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9235c-2125-4cdd-b538-5fcfd7026a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:22.045738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9235c-2125-4cdd-b538-5fcfd7026a45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:22.046017 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9235c-2125-4cdd-b538-5fcfd7026a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:22.046291 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9235c-2125-4cdd-b538-5fcfd7026a45 admin admin] PATH_INFO: `/v3/projects/77cc91a5c0bc4e5888460d7618594895/users/fa949e48248747369eefd74280441b75/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:22.072627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9235c-2125-4cdd-b538-5fcfd7026a45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77cc91a5c0bc4e5888460d7618594895, user_id=fa949e48248747369eefd74280441b75, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:22.077131 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9235c-2125-4cdd-b538-5fcfd7026a45 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:22.100443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1117/2257] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:30:22 2022] PUT /identity/v3/projects/77cc91a5c0bc4e5888460d7618594895/users/fa949e48248747369eefd74280441b75/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:22.108950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811dc656-316d-4eaa-b276-d219d87a2457 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:22.109374 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811dc656-316d-4eaa-b276-d219d87a2457 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:22.109814 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811dc656-316d-4eaa-b276-d219d87a2457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:22.136429 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-811dc656-316d-4eaa-b276-d219d87a2457 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:22.137755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-811dc656-316d-4eaa-b276-d219d87a2457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:22.224360 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1141/2258] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:22.234389 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9235c-2125-4cdd-b538-5fcfd7026a45 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:22.236142 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-95f9235c-2125-4cdd-b538-5fcfd7026a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:22.332767 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9235c-2125-4cdd-b538-5fcfd7026a45 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:22.336845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f1ddb-5dac-475d-be6a-d434e8f5ea2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:22.337857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1ddb-5dac-475d-be6a-d434e8f5ea2a service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:22.337857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1ddb-5dac-475d-be6a-d434e8f5ea2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:22.337857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1ddb-5dac-475d-be6a-d434e8f5ea2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:22.351913 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f1ddb-5dac-475d-be6a-d434e8f5ea2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:22.353684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f1ddb-5dac-475d-be6a-d434e8f5ea2a service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:22.367498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1118/2259] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:30:30.630788 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811dc656-316d-4eaa-b276-d219d87a2457 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:30.634275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-811dc656-316d-4eaa-b276-d219d87a2457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:30.999761 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388bcf7-4593-4c78-abf8-0626f9a834c4 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:31.000623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388bcf7-4593-4c78-abf8-0626f9a834c4 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:31.001096 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388bcf7-4593-4c78-abf8-0626f9a834c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:31.180784 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-3388bcf7-4593-4c78-abf8-0626f9a834c4 None None] MFA Rules not processed for user `fa949e48248747369eefd74280441b75`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:31.187185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3388bcf7-4593-4c78-abf8-0626f9a834c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:31.229972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811dc656-316d-4eaa-b276-d219d87a2457 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:31.257255 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eba9c1-e828-4699-9fb3-dabea663a44d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:31.258916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eba9c1-e828-4699-9fb3-dabea663a44d service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:31.259550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eba9c1-e828-4699-9fb3-dabea663a44d service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:31.260376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eba9c1-e828-4699-9fb3-dabea663a44d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:31.266553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-71eba9c1-e828-4699-9fb3-dabea663a44d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:31.329010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eba9c1-e828-4699-9fb3-dabea663a44d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:31.332261 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eba9c1-e828-4699-9fb3-dabea663a44d service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:31.360218 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1142/2260] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 731 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:30:31.424504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:31.546583 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eba9c1-e828-4699-9fb3-dabea663a44d service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:31.551837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1119/2261] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:30:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 558 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:30:31.606841 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eba9c1-e828-4699-9fb3-dabea663a44d service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:31.629482 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662276e-a09b-40b8-9e9d-db50c9c20058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:31.630329 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662276e-a09b-40b8-9e9d-db50c9c20058 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:31.630621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662276e-a09b-40b8-9e9d-db50c9c20058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:31.630934 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662276e-a09b-40b8-9e9d-db50c9c20058 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:31.632077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662276e-a09b-40b8-9e9d-db50c9c20058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:31.634460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662276e-a09b-40b8-9e9d-db50c9c20058 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:31.708509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1143/2262] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:30:31 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:30:31.714065 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388bcf7-4593-4c78-abf8-0626f9a834c4 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:31.736080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388bcf7-4593-4c78-abf8-0626f9a834c4 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:31.867728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2a29d-413a-44f1-80dd-8d76d93250bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:31.869255 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2a29d-413a-44f1-80dd-8d76d93250bd admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:31.869542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2a29d-413a-44f1-80dd-8d76d93250bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:31.869831 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2a29d-413a-44f1-80dd-8d76d93250bd admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:31.870686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2a29d-413a-44f1-80dd-8d76d93250bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:31.872907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2a29d-413a-44f1-80dd-8d76d93250bd admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:31.907432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1120/2263] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:30:31 2022] POST /identity/v3/users => generated 296 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:31.910978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662276e-a09b-40b8-9e9d-db50c9c20058 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:31.926383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662276e-a09b-40b8-9e9d-db50c9c20058 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:31.934872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607165cf-0f20-4294-a206-08dff91a5fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:31.936364 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607165cf-0f20-4294-a206-08dff91a5fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:31.936659 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607165cf-0f20-4294-a206-08dff91a5fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:31.936936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607165cf-0f20-4294-a206-08dff91a5fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:31.937651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607165cf-0f20-4294-a206-08dff91a5fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:31.939434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607165cf-0f20-4294-a206-08dff91a5fc3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:31.959417 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1144/2264] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:30:31.966274 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2a29d-413a-44f1-80dd-8d76d93250bd admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:31.991260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2a29d-413a-44f1-80dd-8d76d93250bd admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:32.001850 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f602ed-d6d5-4920-8712-bc2f6ca10325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:32.003598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f602ed-d6d5-4920-8712-bc2f6ca10325 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:32.004172 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f602ed-d6d5-4920-8712-bc2f6ca10325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:32.005320 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f602ed-d6d5-4920-8712-bc2f6ca10325 admin admin] PATH_INFO: `/v3/projects/898f0f02fb794dcc96209124944cfe31/users/016968b282b74ecc837fc8756d4be9ee/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:32.035210 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f602ed-d6d5-4920-8712-bc2f6ca10325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=898f0f02fb794dcc96209124944cfe31, user_id=016968b282b74ecc837fc8756d4be9ee, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:32.040828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f602ed-d6d5-4920-8712-bc2f6ca10325 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:32.077115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1121/2265] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:30:31 2022] PUT /identity/v3/projects/898f0f02fb794dcc96209124944cfe31/users/016968b282b74ecc837fc8756d4be9ee/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:30:32.083321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607165cf-0f20-4294-a206-08dff91a5fc3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:32.130998 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607165cf-0f20-4294-a206-08dff91a5fc3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:32.175691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:32.354213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe0169-21b3-4319-8079-0d06289d50e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:32.356890 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe0169-21b3-4319-8079-0d06289d50e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:32.358726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe0169-21b3-4319-8079-0d06289d50e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:32.359442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe0169-21b3-4319-8079-0d06289d50e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:32.361193 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe0169-21b3-4319-8079-0d06289d50e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:32.367625 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe0169-21b3-4319-8079-0d06289d50e2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:32.412872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1145/2266] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:30:32.415152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f602ed-d6d5-4920-8712-bc2f6ca10325 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:32.454245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f602ed-d6d5-4920-8712-bc2f6ca10325 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:32.479326 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c281496-7a2b-41cf-906e-ef5bbcde4d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:32.480550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c281496-7a2b-41cf-906e-ef5bbcde4d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:32.481071 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c281496-7a2b-41cf-906e-ef5bbcde4d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:32.481619 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c281496-7a2b-41cf-906e-ef5bbcde4d60 admin admin] PATH_INFO: `/v3/projects/898f0f02fb794dcc96209124944cfe31/users/016968b282b74ecc837fc8756d4be9ee/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:32.493906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c281496-7a2b-41cf-906e-ef5bbcde4d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=898f0f02fb794dcc96209124944cfe31, user_id=016968b282b74ecc837fc8756d4be9ee, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:32.508534 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c281496-7a2b-41cf-906e-ef5bbcde4d60 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:32.545515 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe0169-21b3-4319-8079-0d06289d50e2 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:32.569336 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1122/2267] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:30:32 2022] PUT /identity/v3/projects/898f0f02fb794dcc96209124944cfe31/users/016968b282b74ecc837fc8756d4be9ee/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:32.763763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe0169-21b3-4319-8079-0d06289d50e2 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:32.999166 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1f034-e781-45ec-9dce-4c6e9eceda54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:33.000362 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1f034-e781-45ec-9dce-4c6e9eceda54 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:33.000950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1f034-e781-45ec-9dce-4c6e9eceda54 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:33.005844 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1f034-e781-45ec-9dce-4c6e9eceda54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:33.010406 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-90e1f034-e781-45ec-9dce-4c6e9eceda54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:33.033319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1f034-e781-45ec-9dce-4c6e9eceda54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:33.045269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1f034-e781-45ec-9dce-4c6e9eceda54 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:33.078632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1146/2268] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:30:42.523674 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=55,keystone:INSERT=5 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:43.054263 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c281496-7a2b-41cf-906e-ef5bbcde4d60 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:43.065772 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=85 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:43.162442 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c281496-7a2b-41cf-906e-ef5bbcde4d60 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:43.173909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2ccb7-91d5-49c5-a9e5-4d63e52a1790 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:43.175318 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ccb7-91d5-49c5-a9e5-4d63e52a1790 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:43.175667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ccb7-91d5-49c5-a9e5-4d63e52a1790 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:43.175945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ccb7-91d5-49c5-a9e5-4d63e52a1790 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:43.178302 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c0b2ccb7-91d5-49c5-a9e5-4d63e52a1790 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:43.250476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2ccb7-91d5-49c5-a9e5-4d63e52a1790 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:43.255391 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2ccb7-91d5-49c5-a9e5-4d63e52a1790 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:43.288793 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1123/2269] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:30:49.904533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9340f65-bb84-410b-96c2-84fda697893c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:49.907693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9340f65-bb84-410b-96c2-84fda697893c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:49.908845 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9340f65-bb84-410b-96c2-84fda697893c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:49.975419 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f9340f65-bb84-410b-96c2-84fda697893c None None] MFA Rules not processed for user `016968b282b74ecc837fc8756d4be9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:49.978311 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f9340f65-bb84-410b-96c2-84fda697893c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:50.201645 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1147/2270] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:30:49 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:50.244932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2ccb7-91d5-49c5-a9e5-4d63e52a1790 service nova] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:50.283579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2ccb7-91d5-49c5-a9e5-4d63e52a1790 service nova] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:50.306531 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2e954-1bf7-440b-90df-d40b9397a5b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:50.307794 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2e954-1bf7-440b-90df-d40b9397a5b5 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:50.315425 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2e954-1bf7-440b-90df-d40b9397a5b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:50.317881 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2e954-1bf7-440b-90df-d40b9397a5b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:50.723725 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2e954-1bf7-440b-90df-d40b9397a5b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:50.727005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2e954-1bf7-440b-90df-d40b9397a5b5 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:50.795679 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1124/2271] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:00.180780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=34 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:00.778781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=56 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:01.278665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9340f65-bb84-410b-96c2-84fda697893c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:01.401862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9340f65-bb84-410b-96c2-84fda697893c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:01.416304 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5449c-59c1-454f-a06c-992c90781677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:01.417248 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5449c-59c1-454f-a06c-992c90781677 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:01.417618 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5449c-59c1-454f-a06c-992c90781677 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:01.417902 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5449c-59c1-454f-a06c-992c90781677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:01.420529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-93c5449c-59c1-454f-a06c-992c90781677 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:01.606727 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5449c-59c1-454f-a06c-992c90781677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:01.608685 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5449c-59c1-454f-a06c-992c90781677 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:01.650997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1148/2272] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:06.860010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a729df3-4545-4d0f-819c-a9a183170bfd None None] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:06.860589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a729df3-4545-4d0f-819c-a9a183170bfd None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:06.860589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a729df3-4545-4d0f-819c-a9a183170bfd None None] PATH_INFO: `/` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:06.862702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1125/2273] 10.222.0.88 () {54 vars in 921 bytes} [Tue Aug 23 22:31:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:31:06.868298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cce36c-0930-458b-94c0-c9bdaaf3a9a3 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:06.868802 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cce36c-0930-458b-94c0-c9bdaaf3a9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:06.868963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cce36c-0930-458b-94c0-c9bdaaf3a9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:06.892241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f2cce36c-0930-458b-94c0-c9bdaaf3a9a3 None None] MFA Rules not processed for user `e642dd7a07ca4afaa40c2769299d8bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:06.893607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f2cce36c-0930-458b-94c0-c9bdaaf3a9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:06.972721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1149/2274] 10.222.0.88 () {58 vars in 1018 bytes} [Tue Aug 23 22:31:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:07.194572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a729df3-4545-4d0f-819c-a9a183170bfd None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:07.196743 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-9a729df3-4545-4d0f-819c-a9a183170bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:07.220059 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a729df3-4545-4d0f-819c-a9a183170bfd None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:07.222319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cce36c-0930-458b-94c0-c9bdaaf3a9a3 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:07.223709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f2cce36c-0930-458b-94c0-c9bdaaf3a9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:07.226629 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf4491-e175-46bb-a9ee-cbd5df77bee0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e642dd7a07ca4afaa40c2769299d8bad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:07.227250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf4491-e175-46bb-a9ee-cbd5df77bee0 service placement] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:07.227605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf4491-e175-46bb-a9ee-cbd5df77bee0 service placement] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:07.227795 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf4491-e175-46bb-a9ee-cbd5df77bee0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:07.241344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cce36c-0930-458b-94c0-c9bdaaf3a9a3 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:07.242497 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf4491-e175-46bb-a9ee-cbd5df77bee0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:07.244132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf4491-e175-46bb-a9ee-cbd5df77bee0 service placement] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:07.253586 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685d478-870e-4cd2-96c8-adf9c3399693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:07.254289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685d478-870e-4cd2-96c8-adf9c3399693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:07.254567 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685d478-870e-4cd2-96c8-adf9c3399693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:07.254846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685d478-870e-4cd2-96c8-adf9c3399693 admin admin] PATH_INFO: `/v3/users/78b05a538ce0466ea8b3e27a0a6fc162` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:07.257128 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685d478-870e-4cd2-96c8-adf9c3399693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b05a538ce0466ea8b3e27a0a6fc162)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:07.259273 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685d478-870e-4cd2-96c8-adf9c3399693 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:07.259410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1126/2275] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:07.261980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:07.340379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b685d478-870e-4cd2-96c8-adf9c3399693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b05a538ce0466ea8b3e27a0a6fc162'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.355337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b685d478-870e-4cd2-96c8-adf9c3399693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b05a538ce0466ea8b3e27a0a6fc162'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.364494 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b685d478-870e-4cd2-96c8-adf9c3399693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b05a538ce0466ea8b3e27a0a6fc162'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.365295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b685d478-870e-4cd2-96c8-adf9c3399693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b05a538ce0466ea8b3e27a0a6fc162'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.373255 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b685d478-870e-4cd2-96c8-adf9c3399693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b05a538ce0466ea8b3e27a0a6fc162'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.380491 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1150/2276] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:07 2022] DELETE /identity/v3/users/78b05a538ce0466ea8b3e27a0a6fc162 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:07.623297 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf4491-e175-46bb-a9ee-cbd5df77bee0 service placement] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:07.637253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf4491-e175-46bb-a9ee-cbd5df77bee0 service placement] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:07.724761 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6065313-fa81-4c67-bb69-3b1937bfc91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:07.725455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6065313-fa81-4c67-bb69-3b1937bfc91b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:07.725760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6065313-fa81-4c67-bb69-3b1937bfc91b admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:07.726035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6065313-fa81-4c67-bb69-3b1937bfc91b admin admin] PATH_INFO: `/v3/projects/1dfc84dc05d542d987194096f3b7e51a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:07.727206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6065313-fa81-4c67-bb69-3b1937bfc91b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dfc84dc05d542d987194096f3b7e51a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:07.729248 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6065313-fa81-4c67-bb69-3b1937bfc91b admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:07.795667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b6065313-fa81-4c67-bb69-3b1937bfc91b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dfc84dc05d542d987194096f3b7e51a', 'request_id': 'req-b6065313-fa81-4c67-bb69-3b1937bfc91b'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.804219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b6065313-fa81-4c67-bb69-3b1937bfc91b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dfc84dc05d542d987194096f3b7e51a', 'request_id': 'req-b6065313-fa81-4c67-bb69-3b1937bfc91b'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.810466 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b6065313-fa81-4c67-bb69-3b1937bfc91b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dfc84dc05d542d987194096f3b7e51a', 'request_id': 'req-b6065313-fa81-4c67-bb69-3b1937bfc91b'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.832113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b6065313-fa81-4c67-bb69-3b1937bfc91b admin admin] The token cache is being invalidate because project 1dfc84dc05d542d987194096f3b7e51a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:07.832484 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b6065313-fa81-4c67-bb69-3b1937bfc91b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.833332 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b6065313-fa81-4c67-bb69-3b1937bfc91b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.834921 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1127/2277] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:07 2022] DELETE /identity/v3/projects/1dfc84dc05d542d987194096f3b7e51a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:07.846406 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af906e9-10b1-453a-947d-c76d5dc549f9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:07.846674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af906e9-10b1-453a-947d-c76d5dc549f9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:07.847077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af906e9-10b1-453a-947d-c76d5dc549f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:07.867064 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2af906e9-10b1-453a-947d-c76d5dc549f9 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:07.868266 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2af906e9-10b1-453a-947d-c76d5dc549f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:07.933079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1151/2278] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:07.942584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefda5cc-1f72-43b1-bb5e-2853cd9ad730 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:07.942584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefda5cc-1f72-43b1-bb5e-2853cd9ad730 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:07.942584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefda5cc-1f72-43b1-bb5e-2853cd9ad730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:07.962464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-fefda5cc-1f72-43b1-bb5e-2853cd9ad730 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:07.963684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-fefda5cc-1f72-43b1-bb5e-2853cd9ad730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:07.971609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1128/2279] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:07.976069 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af906e9-10b1-453a-947d-c76d5dc549f9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:07.986416 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af906e9-10b1-453a-947d-c76d5dc549f9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:07.991849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b45b09-8431-42b6-bd58-ffdb0febca33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:07.992572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b45b09-8431-42b6-bd58-ffdb0febca33 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:07.992991 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b45b09-8431-42b6-bd58-ffdb0febca33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:07.993344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b45b09-8431-42b6-bd58-ffdb0febca33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:07.994041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b45b09-8431-42b6-bd58-ffdb0febca33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:31:07.994383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b45b09-8431-42b6-bd58-ffdb0febca33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:07.996096 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b45b09-8431-42b6-bd58-ffdb0febca33 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:08.009318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1152/2280] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:31:08.014597 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefda5cc-1f72-43b1-bb5e-2853cd9ad730 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:08.026700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefda5cc-1f72-43b1-bb5e-2853cd9ad730 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:08.031440 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc82ea-13a7-483d-a20a-ea88a4ce1606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:08.032042 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc82ea-13a7-483d-a20a-ea88a4ce1606 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:08.032323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc82ea-13a7-483d-a20a-ea88a4ce1606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:08.033207 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc82ea-13a7-483d-a20a-ea88a4ce1606 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:08.034012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc82ea-13a7-483d-a20a-ea88a4ce1606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:08.035951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc82ea-13a7-483d-a20a-ea88a4ce1606 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:08.067796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1129/2281] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:08 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:08.071781 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b45b09-8431-42b6-bd58-ffdb0febca33 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:08.083828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b45b09-8431-42b6-bd58-ffdb0febca33 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:08.144732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289f6d2-f303-4d9b-b4a1-c66789b2263a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:08.145380 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289f6d2-f303-4d9b-b4a1-c66789b2263a admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:08.145665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289f6d2-f303-4d9b-b4a1-c66789b2263a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:08.145937 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289f6d2-f303-4d9b-b4a1-c66789b2263a admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:08.146589 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289f6d2-f303-4d9b-b4a1-c66789b2263a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:08.148784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289f6d2-f303-4d9b-b4a1-c66789b2263a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:08.166389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:08.174713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1153/2282] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:08 2022] POST /identity/v3/users => generated 302 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:08.179349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc82ea-13a7-483d-a20a-ea88a4ce1606 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:08.192303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc82ea-13a7-483d-a20a-ea88a4ce1606 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:08.198096 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0e1ee-25c3-4e7e-9a71-369e811119be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:08.198877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0e1ee-25c3-4e7e-9a71-369e811119be admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:08.207067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0e1ee-25c3-4e7e-9a71-369e811119be admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:08.207067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0e1ee-25c3-4e7e-9a71-369e811119be admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:08.207067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0e1ee-25c3-4e7e-9a71-369e811119be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:08.207067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0e1ee-25c3-4e7e-9a71-369e811119be admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:08.219147 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1130/2283] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:08.224393 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289f6d2-f303-4d9b-b4a1-c66789b2263a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:08.237162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289f6d2-f303-4d9b-b4a1-c66789b2263a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:08.243108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339fbde-00cc-4a5e-877d-fefc2b7e11cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:08.243513 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339fbde-00cc-4a5e-877d-fefc2b7e11cb admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:08.243793 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339fbde-00cc-4a5e-877d-fefc2b7e11cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:08.244049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339fbde-00cc-4a5e-877d-fefc2b7e11cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:08.244709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339fbde-00cc-4a5e-877d-fefc2b7e11cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:08.246449 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339fbde-00cc-4a5e-877d-fefc2b7e11cb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:08.260466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1154/2284] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:08.265086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0e1ee-25c3-4e7e-9a71-369e811119be admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:08.276423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0e1ee-25c3-4e7e-9a71-369e811119be admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:08.281501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aec2e-2780-44e5-a71e-07a8828a1916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:08.282190 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aec2e-2780-44e5-a71e-07a8828a1916 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:08.282487 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aec2e-2780-44e5-a71e-07a8828a1916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:08.282979 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aec2e-2780-44e5-a71e-07a8828a1916 admin admin] PATH_INFO: `/v3/projects/81d16f241a3d4525bd44c53d01e879a8/users/bdc91aadc6204844918f493c95f5d025/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:08.311540 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9aec2e-2780-44e5-a71e-07a8828a1916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81d16f241a3d4525bd44c53d01e879a8, user_id=bdc91aadc6204844918f493c95f5d025, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:08.315067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9aec2e-2780-44e5-a71e-07a8828a1916 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:08.338102 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1131/2285] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:08 2022] PUT /identity/v3/projects/81d16f241a3d4525bd44c53d01e879a8/users/bdc91aadc6204844918f493c95f5d025/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:08.354804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d47b34-6b89-4608-a9c6-40920407b5be None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:08.355401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d47b34-6b89-4608-a9c6-40920407b5be None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:08.355992 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d47b34-6b89-4608-a9c6-40920407b5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:08.403757 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-39d47b34-6b89-4608-a9c6-40920407b5be None None] MFA Rules not processed for user `bdc91aadc6204844918f493c95f5d025`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:08.405177 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-39d47b34-6b89-4608-a9c6-40920407b5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:08.530943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1155/2286] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:08 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:08.537974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aec2e-2780-44e5-a71e-07a8828a1916 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:08.554311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:08.623543 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aec2e-2780-44e5-a71e-07a8828a1916 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:08.629206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75f3c5-bd1c-4c54-909f-d10e483d84dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:08.629874 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75f3c5-bd1c-4c54-909f-d10e483d84dc admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:08.630157 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75f3c5-bd1c-4c54-909f-d10e483d84dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:08.630549 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75f3c5-bd1c-4c54-909f-d10e483d84dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:08.631205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75f3c5-bd1c-4c54-909f-d10e483d84dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:08.634127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75f3c5-bd1c-4c54-909f-d10e483d84dc admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:08.696844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1132/2287] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:08 2022] POST /identity/v3/projects => generated 405 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:08.698787 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d47b34-6b89-4608-a9c6-40920407b5be None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:08.726996 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d47b34-6b89-4608-a9c6-40920407b5be None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:08.905677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b52e1-d096-4fe1-a2ed-14ab1590b903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:08.906682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b52e1-d096-4fe1-a2ed-14ab1590b903 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:08.907214 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b52e1-d096-4fe1-a2ed-14ab1590b903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:08.908559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b52e1-d096-4fe1-a2ed-14ab1590b903 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:08.909442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b52e1-d096-4fe1-a2ed-14ab1590b903 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:08.911715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b52e1-d096-4fe1-a2ed-14ab1590b903 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:08.959256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1156/2288] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:08 2022] POST /identity/v3/users => generated 309 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:08.966370 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75f3c5-bd1c-4c54-909f-d10e483d84dc admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:08.986763 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75f3c5-bd1c-4c54-909f-d10e483d84dc admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:08.992263 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a69f6-5470-4383-8e66-f3945e581722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:08.993231 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a69f6-5470-4383-8e66-f3945e581722 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:08.993826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a69f6-5470-4383-8e66-f3945e581722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:08.994285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a69f6-5470-4383-8e66-f3945e581722 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:08.995377 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a69f6-5470-4383-8e66-f3945e581722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:08.997350 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a69f6-5470-4383-8e66-f3945e581722 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:09.012499 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1133/2289] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:09.017737 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b52e1-d096-4fe1-a2ed-14ab1590b903 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:09.034367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b52e1-d096-4fe1-a2ed-14ab1590b903 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:09.041175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddcd79-1a82-4f62-9877-2c79cae1819e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:09.042382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddcd79-1a82-4f62-9877-2c79cae1819e admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:09.042906 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddcd79-1a82-4f62-9877-2c79cae1819e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:09.043431 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddcd79-1a82-4f62-9877-2c79cae1819e admin admin] PATH_INFO: `/v3/projects/eb61c998b1af4c90a135986b993ec7b3/users/7081c87e48db46d1b0cd91284b34a2b5/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:09.069372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddcd79-1a82-4f62-9877-2c79cae1819e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb61c998b1af4c90a135986b993ec7b3, user_id=7081c87e48db46d1b0cd91284b34a2b5, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:09.072161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddcd79-1a82-4f62-9877-2c79cae1819e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:09.091940 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1157/2290] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:09 2022] PUT /identity/v3/projects/eb61c998b1af4c90a135986b993ec7b3/users/7081c87e48db46d1b0cd91284b34a2b5/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:09.097982 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a69f6-5470-4383-8e66-f3945e581722 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:09.109858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a69f6-5470-4383-8e66-f3945e581722 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:09.186940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4aaef1-abcb-4635-af1e-b1f901805edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:09.187758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4aaef1-abcb-4635-af1e-b1f901805edc admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:09.187945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4aaef1-abcb-4635-af1e-b1f901805edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:09.188259 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4aaef1-abcb-4635-af1e-b1f901805edc admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:09.189340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4aaef1-abcb-4635-af1e-b1f901805edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:09.191539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4aaef1-abcb-4635-af1e-b1f901805edc admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:09.210826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1134/2291] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:09.212112 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddcd79-1a82-4f62-9877-2c79cae1819e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:09.225007 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddcd79-1a82-4f62-9877-2c79cae1819e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:09.232617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa2726-c622-4e9e-bbbc-85c66b6313fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:09.233764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa2726-c622-4e9e-bbbc-85c66b6313fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:09.234198 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa2726-c622-4e9e-bbbc-85c66b6313fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:09.234662 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa2726-c622-4e9e-bbbc-85c66b6313fa admin admin] PATH_INFO: `/v3/projects/eb61c998b1af4c90a135986b993ec7b3/users/7081c87e48db46d1b0cd91284b34a2b5/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:09.238709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa2726-c622-4e9e-bbbc-85c66b6313fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb61c998b1af4c90a135986b993ec7b3, user_id=7081c87e48db46d1b0cd91284b34a2b5, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:09.243022 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa2726-c622-4e9e-bbbc-85c66b6313fa admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:09.256987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1158/2292] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:09 2022] PUT /identity/v3/projects/eb61c998b1af4c90a135986b993ec7b3/users/7081c87e48db46d1b0cd91284b34a2b5/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:09.270884 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98144b14-a4a6-4db8-9cd0-b7515490bc29 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:09.271215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98144b14-a4a6-4db8-9cd0-b7515490bc29 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:09.271578 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98144b14-a4a6-4db8-9cd0-b7515490bc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:09.320590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-98144b14-a4a6-4db8-9cd0-b7515490bc29 None None] MFA Rules not processed for user `7081c87e48db46d1b0cd91284b34a2b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:09.321937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-98144b14-a4a6-4db8-9cd0-b7515490bc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:09.418285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1135/2293] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:09 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:09.430244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa2726-c622-4e9e-bbbc-85c66b6313fa admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:09.432156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d3fa2726-c622-4e9e-bbbc-85c66b6313fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:09.469100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:09.500229 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa2726-c622-4e9e-bbbc-85c66b6313fa admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:09.504415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee319e6f-2fac-43ef-bde6-a82ee20d33fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:09.505017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee319e6f-2fac-43ef-bde6-a82ee20d33fc service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:09.505410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee319e6f-2fac-43ef-bde6-a82ee20d33fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:09.505674 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee319e6f-2fac-43ef-bde6-a82ee20d33fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:09.515142 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee319e6f-2fac-43ef-bde6-a82ee20d33fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:09.516590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee319e6f-2fac-43ef-bde6-a82ee20d33fc service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:09.527248 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1159/2294] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:09 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:31:09.838463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98144b14-a4a6-4db8-9cd0-b7515490bc29 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:09.859326 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:09.927906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98144b14-a4a6-4db8-9cd0-b7515490bc29 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:09.935512 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce1674-219d-4a52-b512-99bcd163d1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:09.936165 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce1674-219d-4a52-b512-99bcd163d1f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:09.936483 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce1674-219d-4a52-b512-99bcd163d1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:09.936730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce1674-219d-4a52-b512-99bcd163d1f3 admin admin] PATH_INFO: `/v3/users/bdc91aadc6204844918f493c95f5d025` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:09.938121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce1674-219d-4a52-b512-99bcd163d1f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc91aadc6204844918f493c95f5d025)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:09.941145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce1674-219d-4a52-b512-99bcd163d1f3 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:10.006600 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-65ce1674-219d-4a52-b512-99bcd163d1f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc91aadc6204844918f493c95f5d025'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:10.008045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-65ce1674-219d-4a52-b512-99bcd163d1f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc91aadc6204844918f493c95f5d025'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:10.016480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-65ce1674-219d-4a52-b512-99bcd163d1f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc91aadc6204844918f493c95f5d025'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:10.037379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-65ce1674-219d-4a52-b512-99bcd163d1f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc91aadc6204844918f493c95f5d025'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:10.049730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-65ce1674-219d-4a52-b512-99bcd163d1f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc91aadc6204844918f493c95f5d025'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:10.057625 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1136/2295] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:09 2022] DELETE /identity/v3/users/bdc91aadc6204844918f493c95f5d025 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:10.062291 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd212592-f271-4e7a-b33a-05f7313785ba None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:10.062806 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd212592-f271-4e7a-b33a-05f7313785ba None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:10.062898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd212592-f271-4e7a-b33a-05f7313785ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:10.085904 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-cd212592-f271-4e7a-b33a-05f7313785ba None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:10.087179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-cd212592-f271-4e7a-b33a-05f7313785ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:10.182240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1160/2296] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:10.197405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce1674-219d-4a52-b512-99bcd163d1f3 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:10.197405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-65ce1674-219d-4a52-b512-99bcd163d1f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:10.297708 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce1674-219d-4a52-b512-99bcd163d1f3 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:10.303013 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807dd6d-200d-4609-8e27-0d55f5aef9dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:10.303867 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807dd6d-200d-4609-8e27-0d55f5aef9dd service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:10.304420 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807dd6d-200d-4609-8e27-0d55f5aef9dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:10.304757 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807dd6d-200d-4609-8e27-0d55f5aef9dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:10.316628 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807dd6d-200d-4609-8e27-0d55f5aef9dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:10.325472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807dd6d-200d-4609-8e27-0d55f5aef9dd service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:10.338531 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1137/2297] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:10.739077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd212592-f271-4e7a-b33a-05f7313785ba None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:10.740788 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-cd212592-f271-4e7a-b33a-05f7313785ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:10.758373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd212592-f271-4e7a-b33a-05f7313785ba None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:10.765329 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce95a32-03db-495a-8c63-7dc26a9bc678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:10.765329 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce95a32-03db-495a-8c63-7dc26a9bc678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:10.765329 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce95a32-03db-495a-8c63-7dc26a9bc678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:10.765329 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce95a32-03db-495a-8c63-7dc26a9bc678 admin admin] PATH_INFO: `/v3/projects/81d16f241a3d4525bd44c53d01e879a8` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:10.766293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce95a32-03db-495a-8c63-7dc26a9bc678 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81d16f241a3d4525bd44c53d01e879a8)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:10.772307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce95a32-03db-495a-8c63-7dc26a9bc678 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:10.834384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ce95a32-03db-495a-8c63-7dc26a9bc678 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81d16f241a3d4525bd44c53d01e879a8', 'request_id': 'req-0ce95a32-03db-495a-8c63-7dc26a9bc678'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:10.846416 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ce95a32-03db-495a-8c63-7dc26a9bc678 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81d16f241a3d4525bd44c53d01e879a8', 'request_id': 'req-0ce95a32-03db-495a-8c63-7dc26a9bc678'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:10.857231 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ce95a32-03db-495a-8c63-7dc26a9bc678 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81d16f241a3d4525bd44c53d01e879a8', 'request_id': 'req-0ce95a32-03db-495a-8c63-7dc26a9bc678'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:10.872751 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ce95a32-03db-495a-8c63-7dc26a9bc678 admin admin] The token cache is being invalidate because project 81d16f241a3d4525bd44c53d01e879a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:10.873351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ce95a32-03db-495a-8c63-7dc26a9bc678 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:10.874106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ce95a32-03db-495a-8c63-7dc26a9bc678 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:10.875623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1161/2298] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:10 2022] DELETE /identity/v3/projects/81d16f241a3d4525bd44c53d01e879a8 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:10.880875 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807dd6d-200d-4609-8e27-0d55f5aef9dd service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:10.893431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807dd6d-200d-4609-8e27-0d55f5aef9dd service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:10.922290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:10.975285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bd4b2-baae-45e4-a184-d85bad144bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:10.975935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bd4b2-baae-45e4-a184-d85bad144bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:10.976423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bd4b2-baae-45e4-a184-d85bad144bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:10.976590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bd4b2-baae-45e4-a184-d85bad144bb4 admin admin] PATH_INFO: `/v3/users/7081c87e48db46d1b0cd91284b34a2b5` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:10.977715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bd4b2-baae-45e4-a184-d85bad144bb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7081c87e48db46d1b0cd91284b34a2b5)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:10.979659 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bd4b2-baae-45e4-a184-d85bad144bb4 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:11.057828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5d7bd4b2-baae-45e4-a184-d85bad144bb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7081c87e48db46d1b0cd91284b34a2b5'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:11.058643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5d7bd4b2-baae-45e4-a184-d85bad144bb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7081c87e48db46d1b0cd91284b34a2b5'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:11.072273 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5d7bd4b2-baae-45e4-a184-d85bad144bb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7081c87e48db46d1b0cd91284b34a2b5'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:11.090702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5d7bd4b2-baae-45e4-a184-d85bad144bb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7081c87e48db46d1b0cd91284b34a2b5'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:11.103915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5d7bd4b2-baae-45e4-a184-d85bad144bb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7081c87e48db46d1b0cd91284b34a2b5'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:11.113014 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1138/2299] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:10 2022] DELETE /identity/v3/users/7081c87e48db46d1b0cd91284b34a2b5 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:11.479909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce95a32-03db-495a-8c63-7dc26a9bc678 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:11.482341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0ce95a32-03db-495a-8c63-7dc26a9bc678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:11.502279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce95a32-03db-495a-8c63-7dc26a9bc678 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:11.603816 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:11.613435 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db16cfe-4455-4f20-812a-36c63b852f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:11.614110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db16cfe-4455-4f20-812a-36c63b852f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:11.626783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db16cfe-4455-4f20-812a-36c63b852f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:11.626783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db16cfe-4455-4f20-812a-36c63b852f5b admin admin] PATH_INFO: `/v3/projects/eb61c998b1af4c90a135986b993ec7b3` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:11.626783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db16cfe-4455-4f20-812a-36c63b852f5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb61c998b1af4c90a135986b993ec7b3)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:11.626783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db16cfe-4455-4f20-812a-36c63b852f5b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:11.681152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8db16cfe-4455-4f20-812a-36c63b852f5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb61c998b1af4c90a135986b993ec7b3', 'request_id': 'req-8db16cfe-4455-4f20-812a-36c63b852f5b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:11.688047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8db16cfe-4455-4f20-812a-36c63b852f5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb61c998b1af4c90a135986b993ec7b3', 'request_id': 'req-8db16cfe-4455-4f20-812a-36c63b852f5b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:11.697157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8db16cfe-4455-4f20-812a-36c63b852f5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb61c998b1af4c90a135986b993ec7b3', 'request_id': 'req-8db16cfe-4455-4f20-812a-36c63b852f5b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:11.712993 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8db16cfe-4455-4f20-812a-36c63b852f5b admin admin] The token cache is being invalidate because project eb61c998b1af4c90a135986b993ec7b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:11.713480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8db16cfe-4455-4f20-812a-36c63b852f5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:11.714173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8db16cfe-4455-4f20-812a-36c63b852f5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:11.715806 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1162/2300] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:11 2022] DELETE /identity/v3/projects/eb61c998b1af4c90a135986b993ec7b3 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:11.721293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e006d5d-dd39-49b5-a71f-3df79403316a None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:11.721595 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e006d5d-dd39-49b5-a71f-3df79403316a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:11.721873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e006d5d-dd39-49b5-a71f-3df79403316a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:11.745487 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-7e006d5d-dd39-49b5-a71f-3df79403316a None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:11.747090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-7e006d5d-dd39-49b5-a71f-3df79403316a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:11.833915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1139/2301] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:11.842193 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2d8be-03e1-41e2-9d3c-38a2aef6c82f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:11.842469 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2d8be-03e1-41e2-9d3c-38a2aef6c82f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:11.842741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2d8be-03e1-41e2-9d3c-38a2aef6c82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:11.865361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-1da2d8be-03e1-41e2-9d3c-38a2aef6c82f None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:11.867564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1da2d8be-03e1-41e2-9d3c-38a2aef6c82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:11.884287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1163/2302] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:11.888100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e006d5d-dd39-49b5-a71f-3df79403316a None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:11.899909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e006d5d-dd39-49b5-a71f-3df79403316a None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:11.907049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94f9cb-e9dd-42e0-943e-b03eed659670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:11.908443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94f9cb-e9dd-42e0-943e-b03eed659670 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:11.908830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94f9cb-e9dd-42e0-943e-b03eed659670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:11.909167 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94f9cb-e9dd-42e0-943e-b03eed659670 admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:11.910039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94f9cb-e9dd-42e0-943e-b03eed659670 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:31:11.910353 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94f9cb-e9dd-42e0-943e-b03eed659670 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:11.912031 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94f9cb-e9dd-42e0-943e-b03eed659670 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:11.927385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1140/2303] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:31:11.932764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2d8be-03e1-41e2-9d3c-38a2aef6c82f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:11.944609 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2d8be-03e1-41e2-9d3c-38a2aef6c82f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:11.950203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36f9c2-7711-461d-b883-e314862733de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:11.950829 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36f9c2-7711-461d-b883-e314862733de admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:11.951113 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36f9c2-7711-461d-b883-e314862733de admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:11.951384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36f9c2-7711-461d-b883-e314862733de admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:11.952034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36f9c2-7711-461d-b883-e314862733de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:11.954042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36f9c2-7711-461d-b883-e314862733de admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:12.009199 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1164/2304] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:11 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:12.010980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94f9cb-e9dd-42e0-943e-b03eed659670 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:12.029966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94f9cb-e9dd-42e0-943e-b03eed659670 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:12.061528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:12.121596 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8d0d8-c2fb-4674-96eb-8e7a9e32c647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:12.122849 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8d0d8-c2fb-4674-96eb-8e7a9e32c647 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.123354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8d0d8-c2fb-4674-96eb-8e7a9e32c647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.123840 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8d0d8-c2fb-4674-96eb-8e7a9e32c647 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.125040 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8d0d8-c2fb-4674-96eb-8e7a9e32c647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:12.129519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8d0d8-c2fb-4674-96eb-8e7a9e32c647 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:12.163114 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1141/2305] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:12 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:12.168760 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36f9c2-7711-461d-b883-e314862733de admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:12.183863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36f9c2-7711-461d-b883-e314862733de admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:12.196085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d66bc1-3a4e-4d55-a807-a14c93a8c224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:12.196769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d66bc1-3a4e-4d55-a807-a14c93a8c224 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.197074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d66bc1-3a4e-4d55-a807-a14c93a8c224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.197407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d66bc1-3a4e-4d55-a807-a14c93a8c224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.198126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d66bc1-3a4e-4d55-a807-a14c93a8c224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:12.199817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d66bc1-3a4e-4d55-a807-a14c93a8c224 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:12.216151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1165/2306] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:12.221287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8d0d8-c2fb-4674-96eb-8e7a9e32c647 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:12.245873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8d0d8-c2fb-4674-96eb-8e7a9e32c647 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:12.254995 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfee7ea-74de-4d09-bbd1-ea6a80d8f024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:12.256065 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfee7ea-74de-4d09-bbd1-ea6a80d8f024 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.256568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfee7ea-74de-4d09-bbd1-ea6a80d8f024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.257029 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfee7ea-74de-4d09-bbd1-ea6a80d8f024 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.258613 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfee7ea-74de-4d09-bbd1-ea6a80d8f024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:12.262091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfee7ea-74de-4d09-bbd1-ea6a80d8f024 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:12.285904 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1142/2307] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:12.292024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d66bc1-3a4e-4d55-a807-a14c93a8c224 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:12.308152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d66bc1-3a4e-4d55-a807-a14c93a8c224 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:12.322698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5061e-f863-43af-bbdd-e21ca31c3c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:12.323398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5061e-f863-43af-bbdd-e21ca31c3c96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.323688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5061e-f863-43af-bbdd-e21ca31c3c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.323964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5061e-f863-43af-bbdd-e21ca31c3c96 admin admin] PATH_INFO: `/v3/projects/9f0b05a9efe14eb78e2edd8f46135229/users/26356b39e36b44b398175f666b7ab555/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.352211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f5061e-f863-43af-bbdd-e21ca31c3c96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f0b05a9efe14eb78e2edd8f46135229, user_id=26356b39e36b44b398175f666b7ab555, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:12.357483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f5061e-f863-43af-bbdd-e21ca31c3c96 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:12.388475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1166/2308] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:12 2022] PUT /identity/v3/projects/9f0b05a9efe14eb78e2edd8f46135229/users/26356b39e36b44b398175f666b7ab555/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:12.409239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121f719-94af-4182-b0ad-1ae6afe1e3b4 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.409694 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121f719-94af-4182-b0ad-1ae6afe1e3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.410164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121f719-94af-4182-b0ad-1ae6afe1e3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.464963 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-e121f719-94af-4182-b0ad-1ae6afe1e3b4 None None] MFA Rules not processed for user `26356b39e36b44b398175f666b7ab555`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:12.466517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e121f719-94af-4182-b0ad-1ae6afe1e3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:12.593716 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1143/2309] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:12.601256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5061e-f863-43af-bbdd-e21ca31c3c96 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:12.715812 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5061e-f863-43af-bbdd-e21ca31c3c96 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:12.725769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd1069-4a9f-4a3a-87fd-182e6422bca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:12.726416 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd1069-4a9f-4a3a-87fd-182e6422bca3 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.726709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd1069-4a9f-4a3a-87fd-182e6422bca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.727039 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd1069-4a9f-4a3a-87fd-182e6422bca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.727735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd1069-4a9f-4a3a-87fd-182e6422bca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:12.730336 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd1069-4a9f-4a3a-87fd-182e6422bca3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:12.753323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121f719-94af-4182-b0ad-1ae6afe1e3b4 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:12.754899 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e121f719-94af-4182-b0ad-1ae6afe1e3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:12.777148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1167/2310] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:12 2022] POST /identity/v3/projects => generated 379 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:12.780088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd1069-4a9f-4a3a-87fd-182e6422bca3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:12.784191 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121f719-94af-4182-b0ad-1ae6afe1e3b4 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:12.799968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:12.875043 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656aba7-cc44-47a2-82af-02d5377478a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:12.876025 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656aba7-cc44-47a2-82af-02d5377478a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.877042 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656aba7-cc44-47a2-82af-02d5377478a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.877541 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656aba7-cc44-47a2-82af-02d5377478a6 admin admin] PATH_INFO: `/v3/users/5918150a21314cf2b28bcee3072457ae` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.879768 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e656aba7-cc44-47a2-82af-02d5377478a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5918150a21314cf2b28bcee3072457ae)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:12.882803 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e656aba7-cc44-47a2-82af-02d5377478a6 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:12.884968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd1069-4a9f-4a3a-87fd-182e6422bca3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:12.894030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d081a-049e-4d95-b6c9-761c5f84f276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:12.894030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d081a-049e-4d95-b6c9-761c5f84f276 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.894030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d081a-049e-4d95-b6c9-761c5f84f276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.894468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d081a-049e-4d95-b6c9-761c5f84f276 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.894982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d081a-049e-4d95-b6c9-761c5f84f276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:12.896923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d081a-049e-4d95-b6c9-761c5f84f276 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:12.922060 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1168/2311] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:12 2022] POST /identity/v3/users => generated 296 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:12.927265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d081a-049e-4d95-b6c9-761c5f84f276 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:12.937604 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:12.938415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d081a-049e-4d95-b6c9-761c5f84f276 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:12.948039 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8fa06-e2a3-47c5-8946-0af80cc73ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:12.948757 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8fa06-e2a3-47c5-8946-0af80cc73ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.949042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8fa06-e2a3-47c5-8946-0af80cc73ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.949365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8fa06-e2a3-47c5-8946-0af80cc73ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.950371 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8fa06-e2a3-47c5-8946-0af80cc73ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:12.952071 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8fa06-e2a3-47c5-8946-0af80cc73ed4 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:12.956277 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e656aba7-cc44-47a2-82af-02d5377478a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5918150a21314cf2b28bcee3072457ae'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:12.957287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e656aba7-cc44-47a2-82af-02d5377478a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5918150a21314cf2b28bcee3072457ae'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:12.968259 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1169/2312] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:12.968461 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e656aba7-cc44-47a2-82af-02d5377478a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5918150a21314cf2b28bcee3072457ae'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:12.970948 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8fa06-e2a3-47c5-8946-0af80cc73ed4 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:12.982022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e656aba7-cc44-47a2-82af-02d5377478a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5918150a21314cf2b28bcee3072457ae'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:12.982435 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8fa06-e2a3-47c5-8946-0af80cc73ed4 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:12.988609 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f0aa7-ed6f-4155-a6f8-4ae4e7cf4918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:12.989311 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f0aa7-ed6f-4155-a6f8-4ae4e7cf4918 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.989998 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f0aa7-ed6f-4155-a6f8-4ae4e7cf4918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.989998 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f0aa7-ed6f-4155-a6f8-4ae4e7cf4918 admin admin] PATH_INFO: `/v3/projects/ae2b8cbd8a7e4c09920122f756ccd720/users/205d2cb3e2ab45ccad90137fcc9d6baa/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.993257 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e656aba7-cc44-47a2-82af-02d5377478a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5918150a21314cf2b28bcee3072457ae'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:13.000417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1144/2313] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:12 2022] DELETE /identity/v3/users/5918150a21314cf2b28bcee3072457ae => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:13.009494 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f0aa7-ed6f-4155-a6f8-4ae4e7cf4918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2b8cbd8a7e4c09920122f756ccd720, user_id=205d2cb3e2ab45ccad90137fcc9d6baa, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:13.012391 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f0aa7-ed6f-4155-a6f8-4ae4e7cf4918 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:13.028219 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1170/2314] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:12 2022] PUT /identity/v3/projects/ae2b8cbd8a7e4c09920122f756ccd720/users/205d2cb3e2ab45ccad90137fcc9d6baa/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:13.032557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656aba7-cc44-47a2-82af-02d5377478a6 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:13.035043 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e656aba7-cc44-47a2-82af-02d5377478a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:13.138644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656aba7-cc44-47a2-82af-02d5377478a6 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:13.238367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0939d9e-25e4-42d9-b250-b0df96566424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:13.239373 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0939d9e-25e4-42d9-b250-b0df96566424 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:13.239655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0939d9e-25e4-42d9-b250-b0df96566424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:13.239929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0939d9e-25e4-42d9-b250-b0df96566424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:13.240580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0939d9e-25e4-42d9-b250-b0df96566424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:13.242587 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0939d9e-25e4-42d9-b250-b0df96566424 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:13.256562 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f0aa7-ed6f-4155-a6f8-4ae4e7cf4918 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:13.258851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ad8f0aa7-ed6f-4155-a6f8-4ae4e7cf4918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:13.260071 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1145/2315] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:13.265978 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0939d9e-25e4-42d9-b250-b0df96566424 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:13.276419 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f0aa7-ed6f-4155-a6f8-4ae4e7cf4918 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:13.281007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0939d9e-25e4-42d9-b250-b0df96566424 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:13.282642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833985a-31d7-4fce-9a89-6c0d5b05ab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:13.283709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833985a-31d7-4fce-9a89-6c0d5b05ab59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:13.284003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833985a-31d7-4fce-9a89-6c0d5b05ab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:13.284439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833985a-31d7-4fce-9a89-6c0d5b05ab59 admin admin] PATH_INFO: `/v3/projects/73aca04a5685447fb698a46da1b0b8ff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:13.285813 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833985a-31d7-4fce-9a89-6c0d5b05ab59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73aca04a5685447fb698a46da1b0b8ff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:13.287678 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2aff7c-cddb-4eef-b97c-88ad1a9bf791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:13.288695 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833985a-31d7-4fce-9a89-6c0d5b05ab59 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:13.289293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2aff7c-cddb-4eef-b97c-88ad1a9bf791 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:13.289851 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2aff7c-cddb-4eef-b97c-88ad1a9bf791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:13.290717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2aff7c-cddb-4eef-b97c-88ad1a9bf791 admin admin] PATH_INFO: `/v3/projects/ae2b8cbd8a7e4c09920122f756ccd720/users/205d2cb3e2ab45ccad90137fcc9d6baa/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:13.294393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2aff7c-cddb-4eef-b97c-88ad1a9bf791 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2b8cbd8a7e4c09920122f756ccd720, user_id=205d2cb3e2ab45ccad90137fcc9d6baa, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:13.299215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2aff7c-cddb-4eef-b97c-88ad1a9bf791 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:13.312671 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1146/2316] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:13 2022] PUT /identity/v3/projects/ae2b8cbd8a7e4c09920122f756ccd720/users/205d2cb3e2ab45ccad90137fcc9d6baa/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:13.324089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0af7d4-1a6e-4bd7-b1d6-4c1741dc2786 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:13.324403 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0af7d4-1a6e-4bd7-b1d6-4c1741dc2786 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:13.324630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0af7d4-1a6e-4bd7-b1d6-4c1741dc2786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:13.354524 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a833985a-31d7-4fce-9a89-6c0d5b05ab59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73aca04a5685447fb698a46da1b0b8ff', 'request_id': 'req-a833985a-31d7-4fce-9a89-6c0d5b05ab59'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:13.358160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-6f0af7d4-1a6e-4bd7-b1d6-4c1741dc2786 None None] MFA Rules not processed for user `205d2cb3e2ab45ccad90137fcc9d6baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:13.359517 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6f0af7d4-1a6e-4bd7-b1d6-4c1741dc2786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:13.361858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a833985a-31d7-4fce-9a89-6c0d5b05ab59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73aca04a5685447fb698a46da1b0b8ff', 'request_id': 'req-a833985a-31d7-4fce-9a89-6c0d5b05ab59'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:13.374084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a833985a-31d7-4fce-9a89-6c0d5b05ab59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73aca04a5685447fb698a46da1b0b8ff', 'request_id': 'req-a833985a-31d7-4fce-9a89-6c0d5b05ab59'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:13.397017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a833985a-31d7-4fce-9a89-6c0d5b05ab59 admin admin] The token cache is being invalidate because project 73aca04a5685447fb698a46da1b0b8ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:13.397394 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a833985a-31d7-4fce-9a89-6c0d5b05ab59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:13.398552 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a833985a-31d7-4fce-9a89-6c0d5b05ab59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:13.400158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1171/2317] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:13 2022] DELETE /identity/v3/projects/73aca04a5685447fb698a46da1b0b8ff => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:13.421094 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:13.489605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1147/2318] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:13.504717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833985a-31d7-4fce-9a89-6c0d5b05ab59 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:13.508640 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a833985a-31d7-4fce-9a89-6c0d5b05ab59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:13.580226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:13.614999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833985a-31d7-4fce-9a89-6c0d5b05ab59 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:13.621763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb40a78-1267-4cee-bff1-19416678acbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:13.622923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb40a78-1267-4cee-bff1-19416678acbd service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:13.623486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb40a78-1267-4cee-bff1-19416678acbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:13.624027 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb40a78-1267-4cee-bff1-19416678acbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:13.642852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb40a78-1267-4cee-bff1-19416678acbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:13.645950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb40a78-1267-4cee-bff1-19416678acbd service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:13.666568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1172/2319] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:13.793747 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0af7d4-1a6e-4bd7-b1d6-4c1741dc2786 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:13.806123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0af7d4-1a6e-4bd7-b1d6-4c1741dc2786 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:13.809769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a13e4d-c7cb-4c23-814e-7fa922d693ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:13.810390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a13e4d-c7cb-4c23-814e-7fa922d693ba service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:13.810668 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a13e4d-c7cb-4c23-814e-7fa922d693ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:13.811098 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a13e4d-c7cb-4c23-814e-7fa922d693ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:13.812994 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-27a13e4d-c7cb-4c23-814e-7fa922d693ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:13.897834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a13e4d-c7cb-4c23-814e-7fa922d693ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:13.899341 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a13e4d-c7cb-4c23-814e-7fa922d693ba service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:13.910847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1148/2320] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:31:14.858052 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e077f-1e59-47dc-93b9-06c76c3bf7cb None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:14.858667 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e077f-1e59-47dc-93b9-06c76c3bf7cb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:14.858667 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e077f-1e59-47dc-93b9-06c76c3bf7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:14.880300 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-270e077f-1e59-47dc-93b9-06c76c3bf7cb None None] MFA Rules not processed for user `fa949e48248747369eefd74280441b75`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:14.881599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-270e077f-1e59-47dc-93b9-06c76c3bf7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:14.974182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1173/2321] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:20.171550 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fde78-3ed6-4e59-9e50-a8694cefd33c None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:20.172191 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fde78-3ed6-4e59-9e50-a8694cefd33c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:20.172699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fde78-3ed6-4e59-9e50-a8694cefd33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:20.223112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-776fde78-3ed6-4e59-9e50-a8694cefd33c None None] MFA Rules not processed for user `1db1a9a77be6452991ec4f99af814e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:20.225596 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-776fde78-3ed6-4e59-9e50-a8694cefd33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:20.328371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1149/2322] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:24.299641 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e077f-1e59-47dc-93b9-06c76c3bf7cb None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:24.302838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-270e077f-1e59-47dc-93b9-06c76c3bf7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:24.521837 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e077f-1e59-47dc-93b9-06c76c3bf7cb None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:24.529203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9d653-2af1-45ad-8aef-f7315a4e56d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:24.530170 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9d653-2af1-45ad-8aef-f7315a4e56d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:24.530529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9d653-2af1-45ad-8aef-f7315a4e56d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:24.530813 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9d653-2af1-45ad-8aef-f7315a4e56d6 admin admin] PATH_INFO: `/v3/users/26356b39e36b44b398175f666b7ab555` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:24.532874 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9d653-2af1-45ad-8aef-f7315a4e56d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26356b39e36b44b398175f666b7ab555)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:24.537509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9d653-2af1-45ad-8aef-f7315a4e56d6 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:24.627798 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-22a9d653-2af1-45ad-8aef-f7315a4e56d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26356b39e36b44b398175f666b7ab555'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:24.653470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-22a9d653-2af1-45ad-8aef-f7315a4e56d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26356b39e36b44b398175f666b7ab555'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:24.665134 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-22a9d653-2af1-45ad-8aef-f7315a4e56d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26356b39e36b44b398175f666b7ab555'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:24.666176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-22a9d653-2af1-45ad-8aef-f7315a4e56d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26356b39e36b44b398175f666b7ab555'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:24.682261 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-22a9d653-2af1-45ad-8aef-f7315a4e56d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26356b39e36b44b398175f666b7ab555'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:24.691946 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1174/2323] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:24 2022] DELETE /identity/v3/users/26356b39e36b44b398175f666b7ab555 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:24.697447 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ffc3d-7a34-4d5b-ad2f-6a6a0b22b87a None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:24.698032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ffc3d-7a34-4d5b-ad2f-6a6a0b22b87a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:24.698456 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ffc3d-7a34-4d5b-ad2f-6a6a0b22b87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:24.734826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-d74ffc3d-7a34-4d5b-ad2f-6a6a0b22b87a None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:24.737157 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d74ffc3d-7a34-4d5b-ad2f-6a6a0b22b87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:24.836047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1150/2324] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:24.852678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9d653-2af1-45ad-8aef-f7315a4e56d6 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:24.854468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-22a9d653-2af1-45ad-8aef-f7315a4e56d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:24.873454 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:24.946262 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9d653-2af1-45ad-8aef-f7315a4e56d6 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:24.953605 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862163c-ac1f-44a4-a030-35670be1dbc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:24.955257 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862163c-ac1f-44a4-a030-35670be1dbc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:24.955882 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862163c-ac1f-44a4-a030-35670be1dbc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:24.956460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862163c-ac1f-44a4-a030-35670be1dbc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:24.976076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862163c-ac1f-44a4-a030-35670be1dbc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:24.979141 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862163c-ac1f-44a4-a030-35670be1dbc6 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:24.999308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1175/2325] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:26.088102 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ffc3d-7a34-4d5b-ad2f-6a6a0b22b87a None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:26.090359 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d74ffc3d-7a34-4d5b-ad2f-6a6a0b22b87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:26.114647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ffc3d-7a34-4d5b-ad2f-6a6a0b22b87a None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:26.135444 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d323bf-cc43-4904-9da3-7a71d5fe7335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:26.136636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d323bf-cc43-4904-9da3-7a71d5fe7335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:26.137172 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d323bf-cc43-4904-9da3-7a71d5fe7335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:26.137720 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d323bf-cc43-4904-9da3-7a71d5fe7335 admin admin] PATH_INFO: `/v3/projects/9f0b05a9efe14eb78e2edd8f46135229` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:26.141107 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d323bf-cc43-4904-9da3-7a71d5fe7335 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f0b05a9efe14eb78e2edd8f46135229)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:26.147069 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d323bf-cc43-4904-9da3-7a71d5fe7335 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:26.170647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:26.260898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f7d323bf-cc43-4904-9da3-7a71d5fe7335 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f0b05a9efe14eb78e2edd8f46135229', 'request_id': 'req-f7d323bf-cc43-4904-9da3-7a71d5fe7335'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:26.273038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f7d323bf-cc43-4904-9da3-7a71d5fe7335 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f0b05a9efe14eb78e2edd8f46135229', 'request_id': 'req-f7d323bf-cc43-4904-9da3-7a71d5fe7335'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:26.281293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f7d323bf-cc43-4904-9da3-7a71d5fe7335 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f0b05a9efe14eb78e2edd8f46135229', 'request_id': 'req-f7d323bf-cc43-4904-9da3-7a71d5fe7335'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:26.304108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f7d323bf-cc43-4904-9da3-7a71d5fe7335 admin admin] The token cache is being invalidate because project 9f0b05a9efe14eb78e2edd8f46135229 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:26.304464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f7d323bf-cc43-4904-9da3-7a71d5fe7335 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:26.314890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f7d323bf-cc43-4904-9da3-7a71d5fe7335 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:26.321159 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1151/2326] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:26 2022] DELETE /identity/v3/projects/9f0b05a9efe14eb78e2edd8f46135229 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:26.325408 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862163c-ac1f-44a4-a030-35670be1dbc6 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:26.341621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862163c-ac1f-44a4-a030-35670be1dbc6 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:26.427867 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf0799-214d-4f9f-ab7c-2ef3aa3cd3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:26.428849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf0799-214d-4f9f-ab7c-2ef3aa3cd3b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:26.429251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf0799-214d-4f9f-ab7c-2ef3aa3cd3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:26.429659 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf0799-214d-4f9f-ab7c-2ef3aa3cd3b7 admin admin] PATH_INFO: `/v3/users/205d2cb3e2ab45ccad90137fcc9d6baa` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:26.431159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf0799-214d-4f9f-ab7c-2ef3aa3cd3b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=205d2cb3e2ab45ccad90137fcc9d6baa)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:26.433471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf0799-214d-4f9f-ab7c-2ef3aa3cd3b7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:26.537479 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-40bf0799-214d-4f9f-ab7c-2ef3aa3cd3b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '205d2cb3e2ab45ccad90137fcc9d6baa'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:26.560127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-40bf0799-214d-4f9f-ab7c-2ef3aa3cd3b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '205d2cb3e2ab45ccad90137fcc9d6baa'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:26.578685 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-40bf0799-214d-4f9f-ab7c-2ef3aa3cd3b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '205d2cb3e2ab45ccad90137fcc9d6baa'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:26.580735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-40bf0799-214d-4f9f-ab7c-2ef3aa3cd3b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '205d2cb3e2ab45ccad90137fcc9d6baa'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:26.602346 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-40bf0799-214d-4f9f-ab7c-2ef3aa3cd3b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '205d2cb3e2ab45ccad90137fcc9d6baa'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:26.615154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1176/2327] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:26 2022] DELETE /identity/v3/users/205d2cb3e2ab45ccad90137fcc9d6baa => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:27.491331 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d323bf-cc43-4904-9da3-7a71d5fe7335 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:27.494203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f7d323bf-cc43-4904-9da3-7a71d5fe7335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:27.523071 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d323bf-cc43-4904-9da3-7a71d5fe7335 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:27.624123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d062e-6eb5-4bdd-9158-2c719c94b195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:27.624123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d062e-6eb5-4bdd-9158-2c719c94b195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:27.624123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d062e-6eb5-4bdd-9158-2c719c94b195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:27.624927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d062e-6eb5-4bdd-9158-2c719c94b195 admin admin] PATH_INFO: `/v3/projects/ae2b8cbd8a7e4c09920122f756ccd720` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:27.626496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d062e-6eb5-4bdd-9158-2c719c94b195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae2b8cbd8a7e4c09920122f756ccd720)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:27.628575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d062e-6eb5-4bdd-9158-2c719c94b195 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:27.694861 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e73d062e-6eb5-4bdd-9158-2c719c94b195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae2b8cbd8a7e4c09920122f756ccd720', 'request_id': 'req-e73d062e-6eb5-4bdd-9158-2c719c94b195'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:27.702432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e73d062e-6eb5-4bdd-9158-2c719c94b195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae2b8cbd8a7e4c09920122f756ccd720', 'request_id': 'req-e73d062e-6eb5-4bdd-9158-2c719c94b195'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:27.712574 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e73d062e-6eb5-4bdd-9158-2c719c94b195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae2b8cbd8a7e4c09920122f756ccd720', 'request_id': 'req-e73d062e-6eb5-4bdd-9158-2c719c94b195'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:27.739616 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e73d062e-6eb5-4bdd-9158-2c719c94b195 admin admin] The token cache is being invalidate because project ae2b8cbd8a7e4c09920122f756ccd720 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:27.740123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e73d062e-6eb5-4bdd-9158-2c719c94b195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:27.741148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-e73d062e-6eb5-4bdd-9158-2c719c94b195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:27.744290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1152/2328] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:27 2022] DELETE /identity/v3/projects/ae2b8cbd8a7e4c09920122f756ccd720 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:27.752398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca22aa-5a2e-4b5c-9269-743ce1c5ea06 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:27.753305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca22aa-5a2e-4b5c-9269-743ce1c5ea06 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:27.753451 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca22aa-5a2e-4b5c-9269-743ce1c5ea06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:27.786469 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-eaca22aa-5a2e-4b5c-9269-743ce1c5ea06 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:27.788527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-eaca22aa-5a2e-4b5c-9269-743ce1c5ea06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:27.843505 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:27.900564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1177/2329] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:27.914125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f415c-f413-4fc7-992a-88c889a08c57 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:27.914579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f415c-f413-4fc7-992a-88c889a08c57 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:27.914936 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f415c-f413-4fc7-992a-88c889a08c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:27.940304 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-019f415c-f413-4fc7-992a-88c889a08c57 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:27.941783 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-019f415c-f413-4fc7-992a-88c889a08c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:27.951363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1153/2330] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:27.957761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca22aa-5a2e-4b5c-9269-743ce1c5ea06 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:27.973093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca22aa-5a2e-4b5c-9269-743ce1c5ea06 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:27.978791 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7ad07-8aa1-41d8-bc83-33ef60c31d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:27.979890 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7ad07-8aa1-41d8-bc83-33ef60c31d80 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:27.980378 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7ad07-8aa1-41d8-bc83-33ef60c31d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:27.980862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7ad07-8aa1-41d8-bc83-33ef60c31d80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:27.982083 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b7ad07-8aa1-41d8-bc83-33ef60c31d80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:31:27.982625 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b7ad07-8aa1-41d8-bc83-33ef60c31d80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:27.986087 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b7ad07-8aa1-41d8-bc83-33ef60c31d80 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:28.009451 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1178/2331] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:31:28.014765 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f415c-f413-4fc7-992a-88c889a08c57 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:28.026206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f415c-f413-4fc7-992a-88c889a08c57 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:28.031149 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eec890-3159-4a6b-b164-6d5ba5bddfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:28.031786 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eec890-3159-4a6b-b164-6d5ba5bddfbf admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:28.032935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eec890-3159-4a6b-b164-6d5ba5bddfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:28.033297 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eec890-3159-4a6b-b164-6d5ba5bddfbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:28.033963 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eec890-3159-4a6b-b164-6d5ba5bddfbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:28.035943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eec890-3159-4a6b-b164-6d5ba5bddfbf admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:28.062749 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:28.073680 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1154/2332] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:28 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:28.080944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7ad07-8aa1-41d8-bc83-33ef60c31d80 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:28.097892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7ad07-8aa1-41d8-bc83-33ef60c31d80 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:28.191702 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584ffa4-7c80-4e71-9b5d-ab3ef490bb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:28.192339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584ffa4-7c80-4e71-9b5d-ab3ef490bb61 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:28.192629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584ffa4-7c80-4e71-9b5d-ab3ef490bb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:28.192900 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584ffa4-7c80-4e71-9b5d-ab3ef490bb61 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:28.193754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584ffa4-7c80-4e71-9b5d-ab3ef490bb61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:28.195650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584ffa4-7c80-4e71-9b5d-ab3ef490bb61 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:28.221363 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1179/2333] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:28 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:28.226025 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eec890-3159-4a6b-b164-6d5ba5bddfbf admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:28.243181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eec890-3159-4a6b-b164-6d5ba5bddfbf admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:28.253086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3ab07-8709-4020-950f-8b494c365b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:28.253724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3ab07-8709-4020-950f-8b494c365b5f admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:28.254006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3ab07-8709-4020-950f-8b494c365b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:28.254282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3ab07-8709-4020-950f-8b494c365b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:28.254960 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3ab07-8709-4020-950f-8b494c365b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:28.256881 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3ab07-8709-4020-950f-8b494c365b5f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:28.277573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1155/2334] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:28.289776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584ffa4-7c80-4e71-9b5d-ab3ef490bb61 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:28.308421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584ffa4-7c80-4e71-9b5d-ab3ef490bb61 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:28.317126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524669fb-ed09-4ac7-b561-df15b2716caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:28.318042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524669fb-ed09-4ac7-b561-df15b2716caa admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:28.318386 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524669fb-ed09-4ac7-b561-df15b2716caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:28.318727 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524669fb-ed09-4ac7-b561-df15b2716caa admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:28.320377 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524669fb-ed09-4ac7-b561-df15b2716caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:28.322240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524669fb-ed09-4ac7-b561-df15b2716caa admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:28.346650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1180/2335] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:28.354400 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3ab07-8709-4020-950f-8b494c365b5f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:28.368456 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3ab07-8709-4020-950f-8b494c365b5f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:28.374802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df878b94-95c7-4bd3-97a8-8758c76da857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:28.375538 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df878b94-95c7-4bd3-97a8-8758c76da857 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:28.376385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df878b94-95c7-4bd3-97a8-8758c76da857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:28.376766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df878b94-95c7-4bd3-97a8-8758c76da857 admin admin] PATH_INFO: `/v3/projects/1ce215fa551d435290e1435c27e268fa/users/1103152693e04fc18922ba3302890e7b/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:28.402117 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df878b94-95c7-4bd3-97a8-8758c76da857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce215fa551d435290e1435c27e268fa, user_id=1103152693e04fc18922ba3302890e7b, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:28.407686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df878b94-95c7-4bd3-97a8-8758c76da857 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:28.432861 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1156/2336] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:28 2022] PUT /identity/v3/projects/1ce215fa551d435290e1435c27e268fa/users/1103152693e04fc18922ba3302890e7b/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:28.447531 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c534d4a-be36-4fee-95c6-50c0bc64e3fd None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:28.448034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c534d4a-be36-4fee-95c6-50c0bc64e3fd None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:28.448524 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c534d4a-be36-4fee-95c6-50c0bc64e3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:28.490604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-5c534d4a-be36-4fee-95c6-50c0bc64e3fd None None] MFA Rules not processed for user `1103152693e04fc18922ba3302890e7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:28.492683 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5c534d4a-be36-4fee-95c6-50c0bc64e3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:28.622045 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1181/2337] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:28.624964 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df878b94-95c7-4bd3-97a8-8758c76da857 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:28.717983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df878b94-95c7-4bd3-97a8-8758c76da857 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:28.730293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7ca40-ccc5-4125-82f1-019bf3e164bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:28.731377 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7ca40-ccc5-4125-82f1-019bf3e164bb admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:28.731695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7ca40-ccc5-4125-82f1-019bf3e164bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:28.731971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7ca40-ccc5-4125-82f1-019bf3e164bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:28.732688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7ca40-ccc5-4125-82f1-019bf3e164bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:28.735187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7ca40-ccc5-4125-82f1-019bf3e164bb admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:28.807496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1157/2338] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:28 2022] POST /identity/v3/projects => generated 383 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:28.815337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c534d4a-be36-4fee-95c6-50c0bc64e3fd None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:28.844401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c534d4a-be36-4fee-95c6-50c0bc64e3fd None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:28.909524 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:28.976232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72699d-0bf0-48a4-bda1-b1155dcb9ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:28.977385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72699d-0bf0-48a4-bda1-b1155dcb9ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:28.977913 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72699d-0bf0-48a4-bda1-b1155dcb9ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:28.978426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72699d-0bf0-48a4-bda1-b1155dcb9ba0 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:28.979588 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72699d-0bf0-48a4-bda1-b1155dcb9ba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:28.983531 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72699d-0bf0-48a4-bda1-b1155dcb9ba0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:29.026348 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1182/2339] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:28 2022] POST /identity/v3/users => generated 298 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:29.031629 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7ca40-ccc5-4125-82f1-019bf3e164bb admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:29.050091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7ca40-ccc5-4125-82f1-019bf3e164bb admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:29.057024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e15c6-593b-48b0-aba2-4205a322cc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:29.058018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e15c6-593b-48b0-aba2-4205a322cc2c admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:29.058424 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e15c6-593b-48b0-aba2-4205a322cc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:29.059003 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e15c6-593b-48b0-aba2-4205a322cc2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:29.060165 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e15c6-593b-48b0-aba2-4205a322cc2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:29.062496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e15c6-593b-48b0-aba2-4205a322cc2c admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:29.084188 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1158/2340] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:29.092938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72699d-0bf0-48a4-bda1-b1155dcb9ba0 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:29.114500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72699d-0bf0-48a4-bda1-b1155dcb9ba0 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:29.123827 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a02777-29a7-4c37-98a5-706e78c8b6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:29.124970 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a02777-29a7-4c37-98a5-706e78c8b6c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:29.125484 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a02777-29a7-4c37-98a5-706e78c8b6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:29.126965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a02777-29a7-4c37-98a5-706e78c8b6c9 admin admin] PATH_INFO: `/v3/projects/aafa45646b66400c8c04daa968d0052c/users/377b88dd79e9455993513f03bb9465e1/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:29.161566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a02777-29a7-4c37-98a5-706e78c8b6c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aafa45646b66400c8c04daa968d0052c, user_id=377b88dd79e9455993513f03bb9465e1, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:29.167451 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a02777-29a7-4c37-98a5-706e78c8b6c9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:29.191539 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1183/2341] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:29 2022] PUT /identity/v3/projects/aafa45646b66400c8c04daa968d0052c/users/377b88dd79e9455993513f03bb9465e1/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:29.198304 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e15c6-593b-48b0-aba2-4205a322cc2c admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:29.216315 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e15c6-593b-48b0-aba2-4205a322cc2c admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:29.329666 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910e716-48a4-4648-b900-32db7f9028ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:29.330538 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910e716-48a4-4648-b900-32db7f9028ce admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:29.330888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910e716-48a4-4648-b900-32db7f9028ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:29.331140 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910e716-48a4-4648-b900-32db7f9028ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:29.331879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7910e716-48a4-4648-b900-32db7f9028ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:29.333818 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7910e716-48a4-4648-b900-32db7f9028ce admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:29.348467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1159/2342] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:29.353356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a02777-29a7-4c37-98a5-706e78c8b6c9 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:29.366594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a02777-29a7-4c37-98a5-706e78c8b6c9 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:29.381340 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfde623-3811-4b1e-9981-3cba4567b1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:29.382052 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfde623-3811-4b1e-9981-3cba4567b1e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:29.382482 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfde623-3811-4b1e-9981-3cba4567b1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:29.382776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfde623-3811-4b1e-9981-3cba4567b1e8 admin admin] PATH_INFO: `/v3/projects/aafa45646b66400c8c04daa968d0052c/users/377b88dd79e9455993513f03bb9465e1/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:29.385209 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfde623-3811-4b1e-9981-3cba4567b1e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aafa45646b66400c8c04daa968d0052c, user_id=377b88dd79e9455993513f03bb9465e1, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:29.388091 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfde623-3811-4b1e-9981-3cba4567b1e8 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:29.404416 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1184/2343] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:29 2022] PUT /identity/v3/projects/aafa45646b66400c8c04daa968d0052c/users/377b88dd79e9455993513f03bb9465e1/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:29.415558 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3b06f-5b69-4fbf-a34e-369d92d6f155 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:29.415918 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3b06f-5b69-4fbf-a34e-369d92d6f155 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:29.416118 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3b06f-5b69-4fbf-a34e-369d92d6f155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:29.450122 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-69d3b06f-5b69-4fbf-a34e-369d92d6f155 None None] MFA Rules not processed for user `377b88dd79e9455993513f03bb9465e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:29.451507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-69d3b06f-5b69-4fbf-a34e-369d92d6f155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:29.478568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:29.546818 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1160/2344] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:29.560258 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfde623-3811-4b1e-9981-3cba4567b1e8 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:29.564025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6dfde623-3811-4b1e-9981-3cba4567b1e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:29.633405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfde623-3811-4b1e-9981-3cba4567b1e8 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:29.637918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3631c-3e10-4adc-b42c-f369d4efee72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:29.638559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3631c-3e10-4adc-b42c-f369d4efee72 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:29.638842 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3631c-3e10-4adc-b42c-f369d4efee72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:29.639122 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3631c-3e10-4adc-b42c-f369d4efee72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:29.649251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3631c-3e10-4adc-b42c-f369d4efee72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:29.650736 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3631c-3e10-4adc-b42c-f369d4efee72 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:29.661793 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1185/2345] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:29.743590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3b06f-5b69-4fbf-a34e-369d92d6f155 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:29.755227 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3b06f-5b69-4fbf-a34e-369d92d6f155 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:29.759354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b48d9-9128-4d81-b81c-68c9e133a4be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:29.759958 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b48d9-9128-4d81-b81c-68c9e133a4be service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:29.760240 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b48d9-9128-4d81-b81c-68c9e133a4be service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:29.760514 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b48d9-9128-4d81-b81c-68c9e133a4be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:29.830813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b48d9-9128-4d81-b81c-68c9e133a4be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:29.832367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b48d9-9128-4d81-b81c-68c9e133a4be service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:29.844658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1161/2346] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:30.163137 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3631c-3e10-4adc-b42c-f369d4efee72 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:30.296309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3631c-3e10-4adc-b42c-f369d4efee72 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:30.307884 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfabbf4-a06a-4ec4-8ee9-e79e7f35d690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:30.308639 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfabbf4-a06a-4ec4-8ee9-e79e7f35d690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:30.308994 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfabbf4-a06a-4ec4-8ee9-e79e7f35d690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:30.309326 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfabbf4-a06a-4ec4-8ee9-e79e7f35d690 admin admin] PATH_INFO: `/v3/users/1103152693e04fc18922ba3302890e7b` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:30.311635 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfabbf4-a06a-4ec4-8ee9-e79e7f35d690 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1103152693e04fc18922ba3302890e7b)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:30.315354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfabbf4-a06a-4ec4-8ee9-e79e7f35d690 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:30.359633 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:30.395078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bbfabbf4-a06a-4ec4-8ee9-e79e7f35d690 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1103152693e04fc18922ba3302890e7b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:30.422254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bbfabbf4-a06a-4ec4-8ee9-e79e7f35d690 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1103152693e04fc18922ba3302890e7b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:30.437682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bbfabbf4-a06a-4ec4-8ee9-e79e7f35d690 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1103152693e04fc18922ba3302890e7b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:30.439892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bbfabbf4-a06a-4ec4-8ee9-e79e7f35d690 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1103152693e04fc18922ba3302890e7b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:30.452058 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bbfabbf4-a06a-4ec4-8ee9-e79e7f35d690 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1103152693e04fc18922ba3302890e7b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:30.462993 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1186/2347] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:30 2022] DELETE /identity/v3/users/1103152693e04fc18922ba3302890e7b => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:30.469231 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf864e-d85f-4b09-ba55-54394b3297f7 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:30.470309 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf864e-d85f-4b09-ba55-54394b3297f7 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:30.470822 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf864e-d85f-4b09-ba55-54394b3297f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:30.509814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-faaf864e-d85f-4b09-ba55-54394b3297f7 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:30.512495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-faaf864e-d85f-4b09-ba55-54394b3297f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:30.644528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1162/2348] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:30.659621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfabbf4-a06a-4ec4-8ee9-e79e7f35d690 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:30.661374 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bbfabbf4-a06a-4ec4-8ee9-e79e7f35d690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:30.780502 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfabbf4-a06a-4ec4-8ee9-e79e7f35d690 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:30.787568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05cf852-fc4a-44e1-a95f-ce4261290c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:30.788570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05cf852-fc4a-44e1-a95f-ce4261290c27 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:30.789135 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05cf852-fc4a-44e1-a95f-ce4261290c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:30.789576 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05cf852-fc4a-44e1-a95f-ce4261290c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:30.805768 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05cf852-fc4a-44e1-a95f-ce4261290c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:30.807678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05cf852-fc4a-44e1-a95f-ce4261290c27 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:30.820741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1187/2349] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:31.306207 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf864e-d85f-4b09-ba55-54394b3297f7 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:31.308186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-faaf864e-d85f-4b09-ba55-54394b3297f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:31.328837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf864e-d85f-4b09-ba55-54394b3297f7 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:31.335896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42295381-e1a9-44b7-9102-8290442ee2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:31.336557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42295381-e1a9-44b7-9102-8290442ee2b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:31.336869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42295381-e1a9-44b7-9102-8290442ee2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:31.337182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42295381-e1a9-44b7-9102-8290442ee2b3 admin admin] PATH_INFO: `/v3/projects/1ce215fa551d435290e1435c27e268fa` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:31.338457 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42295381-e1a9-44b7-9102-8290442ee2b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ce215fa551d435290e1435c27e268fa)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:31.340602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42295381-e1a9-44b7-9102-8290442ee2b3 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:31.420559 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-42295381-e1a9-44b7-9102-8290442ee2b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ce215fa551d435290e1435c27e268fa', 'request_id': 'req-42295381-e1a9-44b7-9102-8290442ee2b3'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:31.424756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:31.435183 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-42295381-e1a9-44b7-9102-8290442ee2b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ce215fa551d435290e1435c27e268fa', 'request_id': 'req-42295381-e1a9-44b7-9102-8290442ee2b3'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:31.449825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-42295381-e1a9-44b7-9102-8290442ee2b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ce215fa551d435290e1435c27e268fa', 'request_id': 'req-42295381-e1a9-44b7-9102-8290442ee2b3'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:31.484626 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-42295381-e1a9-44b7-9102-8290442ee2b3 admin admin] The token cache is being invalidate because project 1ce215fa551d435290e1435c27e268fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:31.485915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-42295381-e1a9-44b7-9102-8290442ee2b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:31.487144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-42295381-e1a9-44b7-9102-8290442ee2b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:31.490756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1163/2350] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:31 2022] DELETE /identity/v3/projects/1ce215fa551d435290e1435c27e268fa => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:31.500014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05cf852-fc4a-44e1-a95f-ce4261290c27 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:31.518331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05cf852-fc4a-44e1-a95f-ce4261290c27 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:31.620726 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3b6d0-50ec-4651-886f-a7f16c1d98d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:31.621825 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3b6d0-50ec-4651-886f-a7f16c1d98d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:31.622292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3b6d0-50ec-4651-886f-a7f16c1d98d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:31.622745 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3b6d0-50ec-4651-886f-a7f16c1d98d5 admin admin] PATH_INFO: `/v3/users/377b88dd79e9455993513f03bb9465e1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:31.624605 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3b6d0-50ec-4651-886f-a7f16c1d98d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=377b88dd79e9455993513f03bb9465e1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:31.627547 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3b6d0-50ec-4651-886f-a7f16c1d98d5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:31.713681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-85a3b6d0-50ec-4651-886f-a7f16c1d98d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '377b88dd79e9455993513f03bb9465e1'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:31.715606 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:31.734908 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-85a3b6d0-50ec-4651-886f-a7f16c1d98d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '377b88dd79e9455993513f03bb9465e1'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:31.747861 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-85a3b6d0-50ec-4651-886f-a7f16c1d98d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '377b88dd79e9455993513f03bb9465e1'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:31.748518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-85a3b6d0-50ec-4651-886f-a7f16c1d98d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '377b88dd79e9455993513f03bb9465e1'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:31.758612 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-85a3b6d0-50ec-4651-886f-a7f16c1d98d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '377b88dd79e9455993513f03bb9465e1'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:31.770647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1188/2351] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:31 2022] DELETE /identity/v3/users/377b88dd79e9455993513f03bb9465e1 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:32.849834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42295381-e1a9-44b7-9102-8290442ee2b3 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:32.854386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-42295381-e1a9-44b7-9102-8290442ee2b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:32.883276 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42295381-e1a9-44b7-9102-8290442ee2b3 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:32.999943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471488c-26ee-437e-a846-c34d7b428c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:33.001965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471488c-26ee-437e-a846-c34d7b428c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:33.002453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471488c-26ee-437e-a846-c34d7b428c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:33.002913 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471488c-26ee-437e-a846-c34d7b428c10 admin admin] PATH_INFO: `/v3/projects/aafa45646b66400c8c04daa968d0052c` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:33.005020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471488c-26ee-437e-a846-c34d7b428c10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aafa45646b66400c8c04daa968d0052c)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:33.008837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471488c-26ee-437e-a846-c34d7b428c10 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:33.111294 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1471488c-26ee-437e-a846-c34d7b428c10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aafa45646b66400c8c04daa968d0052c', 'request_id': 'req-1471488c-26ee-437e-a846-c34d7b428c10'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:33.126844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1471488c-26ee-437e-a846-c34d7b428c10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aafa45646b66400c8c04daa968d0052c', 'request_id': 'req-1471488c-26ee-437e-a846-c34d7b428c10'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:33.138887 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1471488c-26ee-437e-a846-c34d7b428c10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aafa45646b66400c8c04daa968d0052c', 'request_id': 'req-1471488c-26ee-437e-a846-c34d7b428c10'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:33.164991 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1471488c-26ee-437e-a846-c34d7b428c10 admin admin] The token cache is being invalidate because project aafa45646b66400c8c04daa968d0052c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:33.165728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1471488c-26ee-437e-a846-c34d7b428c10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:33.166900 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1471488c-26ee-437e-a846-c34d7b428c10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:33.169653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1164/2352] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:32 2022] DELETE /identity/v3/projects/aafa45646b66400c8c04daa968d0052c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:33.179635 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e719f-e7cc-473f-b03e-1742a8791da1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:33.180422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e719f-e7cc-473f-b03e-1742a8791da1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:33.180608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e719f-e7cc-473f-b03e-1742a8791da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:33.239876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-850e719f-e7cc-473f-b03e-1742a8791da1 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:33.241922 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-850e719f-e7cc-473f-b03e-1742a8791da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:33.354332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1189/2353] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:33.364539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3b0c3-d26b-4989-a46a-e6b9fd5cbd9e None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:33.364824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3b0c3-d26b-4989-a46a-e6b9fd5cbd9e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:33.365149 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3b0c3-d26b-4989-a46a-e6b9fd5cbd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:33.401495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-63a3b0c3-d26b-4989-a46a-e6b9fd5cbd9e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:33.403181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-63a3b0c3-d26b-4989-a46a-e6b9fd5cbd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:33.417132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1165/2354] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:33.422507 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e719f-e7cc-473f-b03e-1742a8791da1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:33.437381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e719f-e7cc-473f-b03e-1742a8791da1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:33.451349 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adfb0c-9f73-4b47-9022-b7749a017467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:33.452018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adfb0c-9f73-4b47-9022-b7749a017467 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:33.452290 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adfb0c-9f73-4b47-9022-b7749a017467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:33.452558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adfb0c-9f73-4b47-9022-b7749a017467 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:33.453468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adfb0c-9f73-4b47-9022-b7749a017467 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:31:33.454793 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adfb0c-9f73-4b47-9022-b7749a017467 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:33.456619 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adfb0c-9f73-4b47-9022-b7749a017467 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:33.489678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1190/2355] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:31:33.493005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3b0c3-d26b-4989-a46a-e6b9fd5cbd9e None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:33.503762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3b0c3-d26b-4989-a46a-e6b9fd5cbd9e None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:33.509833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e83ef4-b6b9-4620-b7fc-cc389f8512c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:33.510896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e83ef4-b6b9-4620-b7fc-cc389f8512c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:33.511346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e83ef4-b6b9-4620-b7fc-cc389f8512c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:33.512101 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e83ef4-b6b9-4620-b7fc-cc389f8512c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:33.513268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e83ef4-b6b9-4620-b7fc-cc389f8512c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:33.517857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e83ef4-b6b9-4620-b7fc-cc389f8512c3 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:33.566352 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1166/2356] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:33 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:33.567520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adfb0c-9f73-4b47-9022-b7749a017467 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:33.579632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adfb0c-9f73-4b47-9022-b7749a017467 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:33.685702 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05969c11-f774-4906-bcab-592b0230f9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:33.686734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05969c11-f774-4906-bcab-592b0230f9ca admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:33.687270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05969c11-f774-4906-bcab-592b0230f9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:33.687767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05969c11-f774-4906-bcab-592b0230f9ca admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:33.689691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05969c11-f774-4906-bcab-592b0230f9ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:33.693509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05969c11-f774-4906-bcab-592b0230f9ca admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:33.729655 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1191/2357] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:33 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:33.734623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e83ef4-b6b9-4620-b7fc-cc389f8512c3 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:33.745740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e83ef4-b6b9-4620-b7fc-cc389f8512c3 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:33.750753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5b7a9-c270-405a-bebf-8cd653213554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:33.751647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5b7a9-c270-405a-bebf-8cd653213554 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:33.752061 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5b7a9-c270-405a-bebf-8cd653213554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:33.752409 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5b7a9-c270-405a-bebf-8cd653213554 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:33.753167 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5b7a9-c270-405a-bebf-8cd653213554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:33.755202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5b7a9-c270-405a-bebf-8cd653213554 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:33.771565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1167/2358] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:33.777176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05969c11-f774-4906-bcab-592b0230f9ca admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:33.789166 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05969c11-f774-4906-bcab-592b0230f9ca admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:33.794259 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44782648-50d3-4a88-bb7e-64df6015adf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:33.794889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44782648-50d3-4a88-bb7e-64df6015adf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:33.795247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44782648-50d3-4a88-bb7e-64df6015adf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:33.795533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44782648-50d3-4a88-bb7e-64df6015adf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:33.796192 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44782648-50d3-4a88-bb7e-64df6015adf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:33.798401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44782648-50d3-4a88-bb7e-64df6015adf2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:33.811224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1192/2359] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:33.816217 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5b7a9-c270-405a-bebf-8cd653213554 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:33.826447 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5b7a9-c270-405a-bebf-8cd653213554 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:33.831981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b4c3a-9b1a-4ecc-a1e0-c7fad38580ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:33.832645 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b4c3a-9b1a-4ecc-a1e0-c7fad38580ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:33.832924 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b4c3a-9b1a-4ecc-a1e0-c7fad38580ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:33.833751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b4c3a-9b1a-4ecc-a1e0-c7fad38580ac admin admin] PATH_INFO: `/v3/projects/3fa39e8a92674aa68edf3c503bfcb735/users/f26132b3b9844b74b6038ee2badd709d/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:33.849108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b4c3a-9b1a-4ecc-a1e0-c7fad38580ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa39e8a92674aa68edf3c503bfcb735, user_id=f26132b3b9844b74b6038ee2badd709d, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:33.851698 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b4c3a-9b1a-4ecc-a1e0-c7fad38580ac admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:33.866204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1168/2360] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:33 2022] PUT /identity/v3/projects/3fa39e8a92674aa68edf3c503bfcb735/users/f26132b3b9844b74b6038ee2badd709d/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:33.877069 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e79772-9466-49a5-b787-fb31c6e03483 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:33.877370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e79772-9466-49a5-b787-fb31c6e03483 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:33.877641 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e79772-9466-49a5-b787-fb31c6e03483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:33.906722 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d4e79772-9466-49a5-b787-fb31c6e03483 None None] MFA Rules not processed for user `f26132b3b9844b74b6038ee2badd709d`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:33.907982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d4e79772-9466-49a5-b787-fb31c6e03483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:33.928279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:33.983141 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1193/2361] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:33.988453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b4c3a-9b1a-4ecc-a1e0-c7fad38580ac admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:33.998341 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:34.059866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b4c3a-9b1a-4ecc-a1e0-c7fad38580ac admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:34.065654 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff42349-99b3-47e7-b16b-fbef2e1d8c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:34.066291 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff42349-99b3-47e7-b16b-fbef2e1d8c29 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:34.066557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff42349-99b3-47e7-b16b-fbef2e1d8c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:34.066834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff42349-99b3-47e7-b16b-fbef2e1d8c29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:34.067534 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff42349-99b3-47e7-b16b-fbef2e1d8c29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:34.069622 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff42349-99b3-47e7-b16b-fbef2e1d8c29 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:34.097464 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1169/2362] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:33 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:34.101966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e79772-9466-49a5-b787-fb31c6e03483 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:34.114023 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e79772-9466-49a5-b787-fb31c6e03483 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:34.174497 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b230d-047c-4206-b85e-80ad31615cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:34.175122 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b230d-047c-4206-b85e-80ad31615cfb admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:34.175441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b230d-047c-4206-b85e-80ad31615cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:34.175721 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b230d-047c-4206-b85e-80ad31615cfb admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:34.176396 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b230d-047c-4206-b85e-80ad31615cfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:34.178756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b230d-047c-4206-b85e-80ad31615cfb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:34.201081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1194/2363] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:34 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:34.205915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff42349-99b3-47e7-b16b-fbef2e1d8c29 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:34.217214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff42349-99b3-47e7-b16b-fbef2e1d8c29 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:34.223650 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adbd15-afb1-4355-8b45-c69bb2770acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:34.224288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adbd15-afb1-4355-8b45-c69bb2770acb admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:34.224565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adbd15-afb1-4355-8b45-c69bb2770acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:34.224838 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adbd15-afb1-4355-8b45-c69bb2770acb admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:34.225555 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adbd15-afb1-4355-8b45-c69bb2770acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:34.227209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adbd15-afb1-4355-8b45-c69bb2770acb admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:34.240786 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1170/2364] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:34.244728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b230d-047c-4206-b85e-80ad31615cfb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:34.258126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b230d-047c-4206-b85e-80ad31615cfb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:34.263745 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ebf6b-7166-4567-9d55-78da951aec56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:34.264853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ebf6b-7166-4567-9d55-78da951aec56 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:34.264853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ebf6b-7166-4567-9d55-78da951aec56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:34.265092 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ebf6b-7166-4567-9d55-78da951aec56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:34.265773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ebf6b-7166-4567-9d55-78da951aec56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:34.268067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ebf6b-7166-4567-9d55-78da951aec56 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:34.283293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1195/2365] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:34.285273 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adbd15-afb1-4355-8b45-c69bb2770acb admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:34.303091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adbd15-afb1-4355-8b45-c69bb2770acb admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:34.310682 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644750f9-03f7-47c5-9bbb-4d2fb45f6763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:34.323800 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644750f9-03f7-47c5-9bbb-4d2fb45f6763 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:34.324177 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644750f9-03f7-47c5-9bbb-4d2fb45f6763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:34.324482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644750f9-03f7-47c5-9bbb-4d2fb45f6763 admin admin] PATH_INFO: `/v3/projects/67e6b1e2f5ea4717b7cfb4d02bb83744/users/0074b868d12449a6919ee9e1a75f32f3/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:34.341600 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644750f9-03f7-47c5-9bbb-4d2fb45f6763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67e6b1e2f5ea4717b7cfb4d02bb83744, user_id=0074b868d12449a6919ee9e1a75f32f3, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:34.344424 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644750f9-03f7-47c5-9bbb-4d2fb45f6763 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:34.362452 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1171/2366] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:34 2022] PUT /identity/v3/projects/67e6b1e2f5ea4717b7cfb4d02bb83744/users/0074b868d12449a6919ee9e1a75f32f3/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:34.373480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d534195-1ddd-442f-a9bd-18988cc85ce4 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:34.373767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d534195-1ddd-442f-a9bd-18988cc85ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:34.374051 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d534195-1ddd-442f-a9bd-18988cc85ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:34.406628 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-3d534195-1ddd-442f-a9bd-18988cc85ce4 None None] MFA Rules not processed for user `0074b868d12449a6919ee9e1a75f32f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:34.407518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3d534195-1ddd-442f-a9bd-18988cc85ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:34.510486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1196/2367] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:34.515583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644750f9-03f7-47c5-9bbb-4d2fb45f6763 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:34.607943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644750f9-03f7-47c5-9bbb-4d2fb45f6763 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:34.618032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce350ae6-e509-4641-b848-96b99e3a5149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:34.618588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce350ae6-e509-4641-b848-96b99e3a5149 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:34.618899 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce350ae6-e509-4641-b848-96b99e3a5149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:34.619184 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce350ae6-e509-4641-b848-96b99e3a5149 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:34.619845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce350ae6-e509-4641-b848-96b99e3a5149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:34.621927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce350ae6-e509-4641-b848-96b99e3a5149 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:34.678142 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1172/2368] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:34 2022] POST /identity/v3/projects => generated 375 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:34.680176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d534195-1ddd-442f-a9bd-18988cc85ce4 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:34.703028 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d534195-1ddd-442f-a9bd-18988cc85ce4 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:34.749927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:34.809409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbff34-7010-41c3-8532-4c9e178f6a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:34.809409 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbff34-7010-41c3-8532-4c9e178f6a5c admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:34.810248 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbff34-7010-41c3-8532-4c9e178f6a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:34.810539 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbff34-7010-41c3-8532-4c9e178f6a5c admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:34.811201 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcbff34-7010-41c3-8532-4c9e178f6a5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:34.814182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcbff34-7010-41c3-8532-4c9e178f6a5c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:34.858089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1197/2369] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:34 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:34.861042 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce350ae6-e509-4641-b848-96b99e3a5149 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:34.880607 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce350ae6-e509-4641-b848-96b99e3a5149 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:34.886439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd40bd1-be42-4a06-b6af-c6b831d4e6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:34.887078 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd40bd1-be42-4a06-b6af-c6b831d4e6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:34.887503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd40bd1-be42-4a06-b6af-c6b831d4e6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:34.887785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd40bd1-be42-4a06-b6af-c6b831d4e6c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:34.888465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd40bd1-be42-4a06-b6af-c6b831d4e6c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:34.891977 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd40bd1-be42-4a06-b6af-c6b831d4e6c4 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:34.920745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1173/2370] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:34.921663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbff34-7010-41c3-8532-4c9e178f6a5c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:34.941913 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbff34-7010-41c3-8532-4c9e178f6a5c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:34.950617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e247a-7593-4558-bbad-c13ff96b2883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:34.951805 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e247a-7593-4558-bbad-c13ff96b2883 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:34.952460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e247a-7593-4558-bbad-c13ff96b2883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:34.952862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e247a-7593-4558-bbad-c13ff96b2883 admin admin] PATH_INFO: `/v3/projects/6746a953978c448dab741d977f0b2989/users/949031e97a80492583e383e89c675f0f/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:34.981514 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e247a-7593-4558-bbad-c13ff96b2883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6746a953978c448dab741d977f0b2989, user_id=949031e97a80492583e383e89c675f0f, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:34.985420 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e247a-7593-4558-bbad-c13ff96b2883 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:35.004997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1198/2371] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:34 2022] PUT /identity/v3/projects/6746a953978c448dab741d977f0b2989/users/949031e97a80492583e383e89c675f0f/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:35.005976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd40bd1-be42-4a06-b6af-c6b831d4e6c4 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:35.019739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd40bd1-be42-4a06-b6af-c6b831d4e6c4 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:35.060424 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:35.098975 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05933dd5-82bd-4e5c-9d93-5f2348869a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:35.099673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05933dd5-82bd-4e5c-9d93-5f2348869a5d admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:35.099997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05933dd5-82bd-4e5c-9d93-5f2348869a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:35.100236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05933dd5-82bd-4e5c-9d93-5f2348869a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:35.100886 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05933dd5-82bd-4e5c-9d93-5f2348869a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:35.102827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05933dd5-82bd-4e5c-9d93-5f2348869a5d admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:35.115148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1174/2372] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:35.119376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e247a-7593-4558-bbad-c13ff96b2883 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:35.132363 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e247a-7593-4558-bbad-c13ff96b2883 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:35.137940 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975ce73-1da9-46cc-8500-0d7150fee5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:35.139055 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975ce73-1da9-46cc-8500-0d7150fee5a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:35.139532 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975ce73-1da9-46cc-8500-0d7150fee5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:35.140004 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975ce73-1da9-46cc-8500-0d7150fee5a3 admin admin] PATH_INFO: `/v3/projects/6746a953978c448dab741d977f0b2989/users/949031e97a80492583e383e89c675f0f/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:35.143594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975ce73-1da9-46cc-8500-0d7150fee5a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6746a953978c448dab741d977f0b2989, user_id=949031e97a80492583e383e89c675f0f, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:35.147249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975ce73-1da9-46cc-8500-0d7150fee5a3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:35.161022 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1199/2373] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:35 2022] PUT /identity/v3/projects/6746a953978c448dab741d977f0b2989/users/949031e97a80492583e383e89c675f0f/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:35.175645 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb26b7-699d-4387-b8f4-95e2fcf7e9d3 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:35.176493 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb26b7-699d-4387-b8f4-95e2fcf7e9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:35.177379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb26b7-699d-4387-b8f4-95e2fcf7e9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:35.222905 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-83eb26b7-699d-4387-b8f4-95e2fcf7e9d3 None None] MFA Rules not processed for user `949031e97a80492583e383e89c675f0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:35.224172 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-83eb26b7-699d-4387-b8f4-95e2fcf7e9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:35.349166 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1175/2374] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:35.363071 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975ce73-1da9-46cc-8500-0d7150fee5a3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:35.365703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3975ce73-1da9-46cc-8500-0d7150fee5a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:35.442691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975ce73-1da9-46cc-8500-0d7150fee5a3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:35.456426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2938417e-c341-48e2-861a-1f4ec9c1e9a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:35.456426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938417e-c341-48e2-861a-1f4ec9c1e9a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:35.456426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938417e-c341-48e2-861a-1f4ec9c1e9a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:35.456426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938417e-c341-48e2-861a-1f4ec9c1e9a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:35.553227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2938417e-c341-48e2-861a-1f4ec9c1e9a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:35.554846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2938417e-c341-48e2-861a-1f4ec9c1e9a3 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:35.566550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1200/2375] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:37.356785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb26b7-699d-4387-b8f4-95e2fcf7e9d3 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:37.368163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb26b7-699d-4387-b8f4-95e2fcf7e9d3 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:37.371925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb939d8-2e79-4eac-aa32-ef4744dee17f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:37.372535 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb939d8-2e79-4eac-aa32-ef4744dee17f service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:37.372808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb939d8-2e79-4eac-aa32-ef4744dee17f service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:37.373099 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb939d8-2e79-4eac-aa32-ef4744dee17f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:37.498158 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb939d8-2e79-4eac-aa32-ef4744dee17f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:37.499656 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb939d8-2e79-4eac-aa32-ef4744dee17f service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:37.511620 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1176/2376] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:38.321535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2938417e-c341-48e2-861a-1f4ec9c1e9a3 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:38.336623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2938417e-c341-48e2-861a-1f4ec9c1e9a3 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:38.350868 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737bee73-68ea-475f-8517-17bb469f4918 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:38.351964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bee73-68ea-475f-8517-17bb469f4918 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:38.352483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bee73-68ea-475f-8517-17bb469f4918 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:38.352961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bee73-68ea-475f-8517-17bb469f4918 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:38.386098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737bee73-68ea-475f-8517-17bb469f4918 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:38.387597 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737bee73-68ea-475f-8517-17bb469f4918 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:38.405566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1201/2377] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:39.503774 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb939d8-2e79-4eac-aa32-ef4744dee17f service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:39.505534 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-fdb939d8-2e79-4eac-aa32-ef4744dee17f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:39.573893 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb939d8-2e79-4eac-aa32-ef4744dee17f service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:39.578877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327641d-7726-416e-ae58-547295c48abb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:39.579890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327641d-7726-416e-ae58-547295c48abb service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:39.580181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327641d-7726-416e-ae58-547295c48abb service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:39.580589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327641d-7726-416e-ae58-547295c48abb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:39.583005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-2327641d-7726-416e-ae58-547295c48abb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:39.589905 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:39.678971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327641d-7726-416e-ae58-547295c48abb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:39.681846 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327641d-7726-416e-ae58-547295c48abb service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:39.699542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1177/2378] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:42.903963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737bee73-68ea-475f-8517-17bb469f4918 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:42.939541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:42.989864 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737bee73-68ea-475f-8517-17bb469f4918 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:42.996841 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1d94b-df17-4f47-a493-1cf5d32985c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:43.005532 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1d94b-df17-4f47-a493-1cf5d32985c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:43.005532 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1d94b-df17-4f47-a493-1cf5d32985c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:43.005532 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1d94b-df17-4f47-a493-1cf5d32985c5 admin admin] PATH_INFO: `/v3/users/f26132b3b9844b74b6038ee2badd709d` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:43.007811 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1d94b-df17-4f47-a493-1cf5d32985c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f26132b3b9844b74b6038ee2badd709d)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:43.010577 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1d94b-df17-4f47-a493-1cf5d32985c5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:43.105851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-add1d94b-df17-4f47-a493-1cf5d32985c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f26132b3b9844b74b6038ee2badd709d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:43.131963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-add1d94b-df17-4f47-a493-1cf5d32985c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f26132b3b9844b74b6038ee2badd709d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:43.157883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-add1d94b-df17-4f47-a493-1cf5d32985c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f26132b3b9844b74b6038ee2badd709d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:43.160437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-add1d94b-df17-4f47-a493-1cf5d32985c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f26132b3b9844b74b6038ee2badd709d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:43.173962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-add1d94b-df17-4f47-a493-1cf5d32985c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f26132b3b9844b74b6038ee2badd709d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:43.194260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf27a59-cf2e-4a7b-a145-ae3a53deb268 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:43.195535 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf27a59-cf2e-4a7b-a145-ae3a53deb268 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:43.196054 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf27a59-cf2e-4a7b-a145-ae3a53deb268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:43.201384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1202/2379] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:42 2022] DELETE /identity/v3/users/f26132b3b9844b74b6038ee2badd709d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:43.244168 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-8cf27a59-cf2e-4a7b-a145-ae3a53deb268 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:43.246184 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8cf27a59-cf2e-4a7b-a145-ae3a53deb268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:43.366508 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1178/2380] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:43.380912 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1d94b-df17-4f47-a493-1cf5d32985c5 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:43.382547 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-add1d94b-df17-4f47-a493-1cf5d32985c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:43.521293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1d94b-df17-4f47-a493-1cf5d32985c5 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:43.527270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c5236-23bb-4759-979b-9f83f5822d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:43.527879 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c5236-23bb-4759-979b-9f83f5822d80 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:43.528359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c5236-23bb-4759-979b-9f83f5822d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:43.528637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c5236-23bb-4759-979b-9f83f5822d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:43.542022 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c5236-23bb-4759-979b-9f83f5822d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:43.543550 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c5236-23bb-4759-979b-9f83f5822d80 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:43.556218 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1203/2381] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:44.126160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf27a59-cf2e-4a7b-a145-ae3a53deb268 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:44.127881 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8cf27a59-cf2e-4a7b-a145-ae3a53deb268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:44.147230 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf27a59-cf2e-4a7b-a145-ae3a53deb268 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:44.152630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057d394-315d-445b-a5a9-a1801b1e9633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:44.153331 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057d394-315d-445b-a5a9-a1801b1e9633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:44.153620 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057d394-315d-445b-a5a9-a1801b1e9633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:44.153909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057d394-315d-445b-a5a9-a1801b1e9633 admin admin] PATH_INFO: `/v3/projects/3fa39e8a92674aa68edf3c503bfcb735` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:44.155245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057d394-315d-445b-a5a9-a1801b1e9633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fa39e8a92674aa68edf3c503bfcb735)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:44.158112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057d394-315d-445b-a5a9-a1801b1e9633 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:44.233511 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b057d394-315d-445b-a5a9-a1801b1e9633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fa39e8a92674aa68edf3c503bfcb735', 'request_id': 'req-b057d394-315d-445b-a5a9-a1801b1e9633'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:44.242543 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b057d394-315d-445b-a5a9-a1801b1e9633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fa39e8a92674aa68edf3c503bfcb735', 'request_id': 'req-b057d394-315d-445b-a5a9-a1801b1e9633'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:44.251037 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b057d394-315d-445b-a5a9-a1801b1e9633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fa39e8a92674aa68edf3c503bfcb735', 'request_id': 'req-b057d394-315d-445b-a5a9-a1801b1e9633'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:44.266660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b057d394-315d-445b-a5a9-a1801b1e9633 admin admin] The token cache is being invalidate because project 3fa39e8a92674aa68edf3c503bfcb735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:44.268138 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b057d394-315d-445b-a5a9-a1801b1e9633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:44.269151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-b057d394-315d-445b-a5a9-a1801b1e9633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:44.270752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1179/2382] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:44 2022] DELETE /identity/v3/projects/3fa39e8a92674aa68edf3c503bfcb735 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:44.275004 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c5236-23bb-4759-979b-9f83f5822d80 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:44.291715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c5236-23bb-4759-979b-9f83f5822d80 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:44.344432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:44.372744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf650b61-028d-4b7a-a534-378c755e263d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:44.373566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf650b61-028d-4b7a-a534-378c755e263d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:44.373854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf650b61-028d-4b7a-a534-378c755e263d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:44.374142 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf650b61-028d-4b7a-a534-378c755e263d admin admin] PATH_INFO: `/v3/users/0074b868d12449a6919ee9e1a75f32f3` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:44.375509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf650b61-028d-4b7a-a534-378c755e263d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0074b868d12449a6919ee9e1a75f32f3)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:44.377663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf650b61-028d-4b7a-a534-378c755e263d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:44.437345 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-cf650b61-028d-4b7a-a534-378c755e263d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0074b868d12449a6919ee9e1a75f32f3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:44.453841 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-cf650b61-028d-4b7a-a534-378c755e263d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0074b868d12449a6919ee9e1a75f32f3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:44.465157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-cf650b61-028d-4b7a-a534-378c755e263d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0074b868d12449a6919ee9e1a75f32f3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:44.465941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-cf650b61-028d-4b7a-a534-378c755e263d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0074b868d12449a6919ee9e1a75f32f3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:44.475148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-cf650b61-028d-4b7a-a534-378c755e263d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0074b868d12449a6919ee9e1a75f32f3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:44.482475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1204/2383] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:44 2022] DELETE /identity/v3/users/0074b868d12449a6919ee9e1a75f32f3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:44.885926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057d394-315d-445b-a5a9-a1801b1e9633 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:44.888101 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-b057d394-315d-445b-a5a9-a1801b1e9633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:44.912887 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057d394-315d-445b-a5a9-a1801b1e9633 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:44.971758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:45.028630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:45.029439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:45.029639 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:45.029922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f admin admin] PATH_INFO: `/v3/projects/67e6b1e2f5ea4717b7cfb4d02bb83744` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:45.032648 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f admin admin] RBAC: Authorizing `identity:delete_project(project_id=67e6b1e2f5ea4717b7cfb4d02bb83744)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:45.035391 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:45.103966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67e6b1e2f5ea4717b7cfb4d02bb83744', 'request_id': 'req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.113154 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67e6b1e2f5ea4717b7cfb4d02bb83744', 'request_id': 'req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.120217 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67e6b1e2f5ea4717b7cfb4d02bb83744', 'request_id': 'req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.139679 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f admin admin] The token cache is being invalidate because project 67e6b1e2f5ea4717b7cfb4d02bb83744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:45.140070 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.141238 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.143290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1180/2384] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:44 2022] DELETE /identity/v3/projects/67e6b1e2f5ea4717b7cfb4d02bb83744 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:45.149195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf650b61-028d-4b7a-a534-378c755e263d admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:45.151264 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-cf650b61-028d-4b7a-a534-378c755e263d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:45.166749 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf650b61-028d-4b7a-a534-378c755e263d admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:45.249837 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06206c5-a17a-4187-bbe4-a062b0ab19a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:45.251348 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06206c5-a17a-4187-bbe4-a062b0ab19a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:45.251858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06206c5-a17a-4187-bbe4-a062b0ab19a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:45.252310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06206c5-a17a-4187-bbe4-a062b0ab19a2 admin admin] PATH_INFO: `/v3/users/949031e97a80492583e383e89c675f0f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:45.254568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06206c5-a17a-4187-bbe4-a062b0ab19a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=949031e97a80492583e383e89c675f0f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:45.258365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06206c5-a17a-4187-bbe4-a062b0ab19a2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:45.360911 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c06206c5-a17a-4187-bbe4-a062b0ab19a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '949031e97a80492583e383e89c675f0f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.387454 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c06206c5-a17a-4187-bbe4-a062b0ab19a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '949031e97a80492583e383e89c675f0f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.391756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:45.404529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c06206c5-a17a-4187-bbe4-a062b0ab19a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '949031e97a80492583e383e89c675f0f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.406608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c06206c5-a17a-4187-bbe4-a062b0ab19a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '949031e97a80492583e383e89c675f0f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.419767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c06206c5-a17a-4187-bbe4-a062b0ab19a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '949031e97a80492583e383e89c675f0f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.434551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1205/2385] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:45 2022] DELETE /identity/v3/users/949031e97a80492583e383e89c675f0f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:45.852495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:45.855382 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:45.877395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc8ddc-d59a-4315-90f6-720a87d2b25f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:45.950110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dbbc7-4476-4db5-b1ea-bbe08e69673e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:45.950799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dbbc7-4476-4db5-b1ea-bbe08e69673e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:45.951285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dbbc7-4476-4db5-b1ea-bbe08e69673e admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:45.951752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dbbc7-4476-4db5-b1ea-bbe08e69673e admin admin] PATH_INFO: `/v3/projects/6746a953978c448dab741d977f0b2989` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:45.954179 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9dbbc7-4476-4db5-b1ea-bbe08e69673e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6746a953978c448dab741d977f0b2989)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:45.956049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9dbbc7-4476-4db5-b1ea-bbe08e69673e admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:46.036075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-ed9dbbc7-4476-4db5-b1ea-bbe08e69673e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6746a953978c448dab741d977f0b2989', 'request_id': 'req-ed9dbbc7-4476-4db5-b1ea-bbe08e69673e'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:46.050233 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-ed9dbbc7-4476-4db5-b1ea-bbe08e69673e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6746a953978c448dab741d977f0b2989', 'request_id': 'req-ed9dbbc7-4476-4db5-b1ea-bbe08e69673e'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:46.055288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:46.062257 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-ed9dbbc7-4476-4db5-b1ea-bbe08e69673e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6746a953978c448dab741d977f0b2989', 'request_id': 'req-ed9dbbc7-4476-4db5-b1ea-bbe08e69673e'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:46.080598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-ed9dbbc7-4476-4db5-b1ea-bbe08e69673e admin admin] The token cache is being invalidate because project 6746a953978c448dab741d977f0b2989 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:46.080980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-ed9dbbc7-4476-4db5-b1ea-bbe08e69673e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:46.081911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-ed9dbbc7-4476-4db5-b1ea-bbe08e69673e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:46.084012 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1181/2386] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:45 2022] DELETE /identity/v3/projects/6746a953978c448dab741d977f0b2989 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:46.090623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363256da-c8c7-4caa-a5df-b95baea51e77 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:46.091063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363256da-c8c7-4caa-a5df-b95baea51e77 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:46.091470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363256da-c8c7-4caa-a5df-b95baea51e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:46.116190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-363256da-c8c7-4caa-a5df-b95baea51e77 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:46.118934 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-363256da-c8c7-4caa-a5df-b95baea51e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:46.226294 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1206/2387] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:46.231383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff93a2-e6df-4b34-b2f8-11e1b023dde2 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:46.231887 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff93a2-e6df-4b34-b2f8-11e1b023dde2 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:46.232385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff93a2-e6df-4b34-b2f8-11e1b023dde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:46.271219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-55ff93a2-e6df-4b34-b2f8-11e1b023dde2 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:46.272703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-55ff93a2-e6df-4b34-b2f8-11e1b023dde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:46.282813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1182/2388] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:46.288131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363256da-c8c7-4caa-a5df-b95baea51e77 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:46.302047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363256da-c8c7-4caa-a5df-b95baea51e77 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:46.309638 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc5610-fd5e-4b22-a1b4-f66eb26c65f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:46.310706 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc5610-fd5e-4b22-a1b4-f66eb26c65f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:46.311226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc5610-fd5e-4b22-a1b4-f66eb26c65f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:46.311747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc5610-fd5e-4b22-a1b4-f66eb26c65f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:46.313506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc5610-fd5e-4b22-a1b4-f66eb26c65f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:31:46.314095 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc5610-fd5e-4b22-a1b4-f66eb26c65f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:46.316079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc5610-fd5e-4b22-a1b4-f66eb26c65f9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:46.340579 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1207/2389] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:31:46.344209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff93a2-e6df-4b34-b2f8-11e1b023dde2 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:46.361225 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff93a2-e6df-4b34-b2f8-11e1b023dde2 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:46.369169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885aa39-1388-40a7-823c-9e0737fac902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:46.370136 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885aa39-1388-40a7-823c-9e0737fac902 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:46.370607 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885aa39-1388-40a7-823c-9e0737fac902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:46.370943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885aa39-1388-40a7-823c-9e0737fac902 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:46.371972 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885aa39-1388-40a7-823c-9e0737fac902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:46.376305 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885aa39-1388-40a7-823c-9e0737fac902 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:46.419286 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1183/2390] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:46 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:46.428163 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc5610-fd5e-4b22-a1b4-f66eb26c65f9 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:46.447277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc5610-fd5e-4b22-a1b4-f66eb26c65f9 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:46.582391 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a3cfe-347b-4d83-b8f8-e9fa1fca0c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:46.583865 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a3cfe-347b-4d83-b8f8-e9fa1fca0c4d admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:46.584406 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a3cfe-347b-4d83-b8f8-e9fa1fca0c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:46.584896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a3cfe-347b-4d83-b8f8-e9fa1fca0c4d admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:46.586249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a3cfe-347b-4d83-b8f8-e9fa1fca0c4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:46.589498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a3cfe-347b-4d83-b8f8-e9fa1fca0c4d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:46.626777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1208/2391] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:46 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:46.630429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885aa39-1388-40a7-823c-9e0737fac902 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:46.651059 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885aa39-1388-40a7-823c-9e0737fac902 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:46.660519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb478c-f228-4111-b76b-32c892ba89c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:46.661620 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb478c-f228-4111-b76b-32c892ba89c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:46.662125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb478c-f228-4111-b76b-32c892ba89c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:46.662624 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb478c-f228-4111-b76b-32c892ba89c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:46.663870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb478c-f228-4111-b76b-32c892ba89c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:46.667772 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb478c-f228-4111-b76b-32c892ba89c6 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:46.694906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1184/2392] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:46.696235 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a3cfe-347b-4d83-b8f8-e9fa1fca0c4d admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:46.709134 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a3cfe-347b-4d83-b8f8-e9fa1fca0c4d admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:46.723434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace99e8e-8931-498d-913e-0fbb54434bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:46.724497 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace99e8e-8931-498d-913e-0fbb54434bfc admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:46.724949 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace99e8e-8931-498d-913e-0fbb54434bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:46.725653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace99e8e-8931-498d-913e-0fbb54434bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:46.726632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace99e8e-8931-498d-913e-0fbb54434bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:46.729532 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace99e8e-8931-498d-913e-0fbb54434bfc admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:46.751762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1209/2393] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:46.757094 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb478c-f228-4111-b76b-32c892ba89c6 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:46.773591 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb478c-f228-4111-b76b-32c892ba89c6 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:46.779260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9d019-f9b0-4415-84b5-6ac5b70106e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:46.780050 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9d019-f9b0-4415-84b5-6ac5b70106e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:46.780455 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9d019-f9b0-4415-84b5-6ac5b70106e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:46.780931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9d019-f9b0-4415-84b5-6ac5b70106e0 admin admin] PATH_INFO: `/v3/projects/566fc3fbaf4742f2a03c2eb258adfb46/users/4f2c5be59eb64eecae67015ac9bfcb83/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:46.809740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9d019-f9b0-4415-84b5-6ac5b70106e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=566fc3fbaf4742f2a03c2eb258adfb46, user_id=4f2c5be59eb64eecae67015ac9bfcb83, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:46.815076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9d019-f9b0-4415-84b5-6ac5b70106e0 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:46.837423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1185/2394] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:46 2022] PUT /identity/v3/projects/566fc3fbaf4742f2a03c2eb258adfb46/users/4f2c5be59eb64eecae67015ac9bfcb83/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:46.912501 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f4e3f-50b7-4c07-8a7e-fe817f880a9e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:46.912918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f4e3f-50b7-4c07-8a7e-fe817f880a9e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:46.913448 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f4e3f-50b7-4c07-8a7e-fe817f880a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:46.951304 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-087f4e3f-50b7-4c07-8a7e-fe817f880a9e None None] MFA Rules not processed for user `4f2c5be59eb64eecae67015ac9bfcb83`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:46.953000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-087f4e3f-50b7-4c07-8a7e-fe817f880a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:47.007584 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:47.057487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1210/2395] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:47.075832 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9d019-f9b0-4415-84b5-6ac5b70106e0 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:47.078736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-92c9d019-f9b0-4415-84b5-6ac5b70106e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:47.180790 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9d019-f9b0-4415-84b5-6ac5b70106e0 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:47.190872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ac356-0abc-463e-9903-483c7c72e08a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:47.192338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ac356-0abc-463e-9903-483c7c72e08a service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:47.193144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ac356-0abc-463e-9903-483c7c72e08a service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:47.193708 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ac356-0abc-463e-9903-483c7c72e08a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:47.212672 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ac356-0abc-463e-9903-483c7c72e08a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:47.214833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ac356-0abc-463e-9903-483c7c72e08a service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:47.233513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1186/2396] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:50.126789 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f4e3f-50b7-4c07-8a7e-fe817f880a9e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:50.215958 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f4e3f-50b7-4c07-8a7e-fe817f880a9e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:50.225200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd01dd-4562-4a4a-b816-be251fec3719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:50.226111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd01dd-4562-4a4a-b816-be251fec3719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:50.226593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd01dd-4562-4a4a-b816-be251fec3719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:50.227097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd01dd-4562-4a4a-b816-be251fec3719 admin admin] PATH_INFO: `/v3/users/4f2c5be59eb64eecae67015ac9bfcb83` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:50.229347 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd01dd-4562-4a4a-b816-be251fec3719 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f2c5be59eb64eecae67015ac9bfcb83)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:50.233286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd01dd-4562-4a4a-b816-be251fec3719 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:50.307704 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c4fd01dd-4562-4a4a-b816-be251fec3719 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f2c5be59eb64eecae67015ac9bfcb83'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:50.335759 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c4fd01dd-4562-4a4a-b816-be251fec3719 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f2c5be59eb64eecae67015ac9bfcb83'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:50.354418 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c4fd01dd-4562-4a4a-b816-be251fec3719 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f2c5be59eb64eecae67015ac9bfcb83'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:50.356025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c4fd01dd-4562-4a4a-b816-be251fec3719 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f2c5be59eb64eecae67015ac9bfcb83'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:50.372238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c4fd01dd-4562-4a4a-b816-be251fec3719 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f2c5be59eb64eecae67015ac9bfcb83'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:50.389102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1211/2397] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:50 2022] DELETE /identity/v3/users/4f2c5be59eb64eecae67015ac9bfcb83 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:50.393802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4cc3c-988d-49cd-b5de-7bf4896f17ce None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:50.394589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4cc3c-988d-49cd-b5de-7bf4896f17ce None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:50.394589 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4cc3c-988d-49cd-b5de-7bf4896f17ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:50.420663 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-3ea4cc3c-988d-49cd-b5de-7bf4896f17ce None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:50.421911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3ea4cc3c-988d-49cd-b5de-7bf4896f17ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:50.550798 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1187/2398] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:50.568796 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd01dd-4562-4a4a-b816-be251fec3719 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:50.571540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c4fd01dd-4562-4a4a-b816-be251fec3719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:50.669803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd01dd-4562-4a4a-b816-be251fec3719 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:50.675275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568bb75-b0d1-4f6d-921a-d8a542e8e3fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:50.676098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568bb75-b0d1-4f6d-921a-d8a542e8e3fa service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:50.676756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568bb75-b0d1-4f6d-921a-d8a542e8e3fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:50.677203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568bb75-b0d1-4f6d-921a-d8a542e8e3fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:50.695540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568bb75-b0d1-4f6d-921a-d8a542e8e3fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:50.698865 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568bb75-b0d1-4f6d-921a-d8a542e8e3fa service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:50.711342 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:50.730088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1212/2399] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:51.136280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed279c35-03dd-4117-b48e-b936cca4aca6 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:51.136616 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed279c35-03dd-4117-b48e-b936cca4aca6 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:51.137156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed279c35-03dd-4117-b48e-b936cca4aca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:51.173135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-ed279c35-03dd-4117-b48e-b936cca4aca6 None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:51.174857 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ed279c35-03dd-4117-b48e-b936cca4aca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:51.209843 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:51.287024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1188/2400] 10.222.0.88 () {58 vars in 1009 bytes} [Tue Aug 23 22:31:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:51.378650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568bb75-b0d1-4f6d-921a-d8a542e8e3fa service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:51.381024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a568bb75-b0d1-4f6d-921a-d8a542e8e3fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:51.421599 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed279c35-03dd-4117-b48e-b936cca4aca6 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:51.424454 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ed279c35-03dd-4117-b48e-b936cca4aca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:51.456179 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed279c35-03dd-4117-b48e-b936cca4aca6 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:51.478858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:51.479987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:51.480539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:51.480980 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66 admin admin] PATH_INFO: `/v3/projects/566fc3fbaf4742f2a03c2eb258adfb46` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:51.482745 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=566fc3fbaf4742f2a03c2eb258adfb46)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:51.486749 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:51.531784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568bb75-b0d1-4f6d-921a-d8a542e8e3fa service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:51.544296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af0be4-45ae-4f45-8346-1092547dfb30 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:51.545115 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af0be4-45ae-4f45-8346-1092547dfb30 service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:51.545576 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af0be4-45ae-4f45-8346-1092547dfb30 service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:51.545892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af0be4-45ae-4f45-8346-1092547dfb30 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:51.558113 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af0be4-45ae-4f45-8346-1092547dfb30 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:51.559965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af0be4-45ae-4f45-8346-1092547dfb30 service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:51.573898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1213/2401] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:31:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:31:51.576068 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '566fc3fbaf4742f2a03c2eb258adfb46', 'request_id': 'req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:51.587870 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '566fc3fbaf4742f2a03c2eb258adfb46', 'request_id': 'req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:51.610895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '566fc3fbaf4742f2a03c2eb258adfb46', 'request_id': 'req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:51.706415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66 admin admin] The token cache is being invalidate because project 566fc3fbaf4742f2a03c2eb258adfb46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:51.709268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:51.714267 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:51.721970 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1189/2402] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:51 2022] DELETE /identity/v3/projects/566fc3fbaf4742f2a03c2eb258adfb46 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:58.380366 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af0be4-45ae-4f45-8346-1092547dfb30 service swift] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:58.383138 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-00af0be4-45ae-4f45-8346-1092547dfb30 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:58.399492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af0be4-45ae-4f45-8346-1092547dfb30 service swift] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:58.511612 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8024d-af69-4897-9a33-1e5d323cf41c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:58.512147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8024d-af69-4897-9a33-1e5d323cf41c service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:58.512433 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8024d-af69-4897-9a33-1e5d323cf41c service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:58.512716 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8024d-af69-4897-9a33-1e5d323cf41c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:58.514593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-34b8024d-af69-4897-9a33-1e5d323cf41c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:58.612864 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8024d-af69-4897-9a33-1e5d323cf41c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:58.614392 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8024d-af69-4897-9a33-1e5d323cf41c service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:58.631803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1214/2403] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:01.660967 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:03.732778 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:03.737757 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:03.886607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8024d-af69-4897-9a33-1e5d323cf41c service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:03.889272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-34b8024d-af69-4897-9a33-1e5d323cf41c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:03.920946 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8024d-af69-4897-9a33-1e5d323cf41c service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:03.934876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bc0dd-87a6-4c9c-a148-e904b9c5bf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:03.935717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bc0dd-87a6-4c9c-a148-e904b9c5bf42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:03.936087 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bc0dd-87a6-4c9c-a148-e904b9c5bf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:03.938957 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bc0dd-87a6-4c9c-a148-e904b9c5bf42 admin admin] PATH_INFO: `/v3/users/e720fd25d54b4dc6b3387ea762d7e4ae` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:03.940905 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8bc0dd-87a6-4c9c-a148-e904b9c5bf42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e720fd25d54b4dc6b3387ea762d7e4ae)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:03.944993 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8bc0dd-87a6-4c9c-a148-e904b9c5bf42 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:03.952032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d8c6b-f1bd-47af-a954-49e97c8ceb66 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:03.965022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23266606-2287-4b16-9322-9ed817a0c469 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e642dd7a07ca4afaa40c2769299d8bad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:03.966735 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23266606-2287-4b16-9322-9ed817a0c469 service placement] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:03.967486 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23266606-2287-4b16-9322-9ed817a0c469 service placement] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:03.968965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23266606-2287-4b16-9322-9ed817a0c469 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:03.974157 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-23266606-2287-4b16-9322-9ed817a0c469 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:03.999246 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23266606-2287-4b16-9322-9ed817a0c469 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:04.004562 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23266606-2287-4b16-9322-9ed817a0c469 service placement] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:04.026416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1190/2404] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:04.029278 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:04.050469 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5d8bc0dd-87a6-4c9c-a148-e904b9c5bf42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e720fd25d54b4dc6b3387ea762d7e4ae'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:04.086697 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5d8bc0dd-87a6-4c9c-a148-e904b9c5bf42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e720fd25d54b4dc6b3387ea762d7e4ae'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:04.103441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5d8bc0dd-87a6-4c9c-a148-e904b9c5bf42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e720fd25d54b4dc6b3387ea762d7e4ae'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:04.104555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5d8bc0dd-87a6-4c9c-a148-e904b9c5bf42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e720fd25d54b4dc6b3387ea762d7e4ae'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:04.122852 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5d8bc0dd-87a6-4c9c-a148-e904b9c5bf42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e720fd25d54b4dc6b3387ea762d7e4ae'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:04.135924 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1215/2405] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:03 2022] DELETE /identity/v3/users/e720fd25d54b4dc6b3387ea762d7e4ae => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:04.438419 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23266606-2287-4b16-9322-9ed817a0c469 service placement] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:04.460167 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23266606-2287-4b16-9322-9ed817a0c469 service placement] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:04.579068 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74a704-c572-45c2-bd9f-6c55deb16d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:04.580001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74a704-c572-45c2-bd9f-6c55deb16d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:04.580400 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74a704-c572-45c2-bd9f-6c55deb16d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:04.581044 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74a704-c572-45c2-bd9f-6c55deb16d01 admin admin] PATH_INFO: `/v3/projects/a297ef598b76404b86e9350e120b86c9` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:04.585112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74a704-c572-45c2-bd9f-6c55deb16d01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a297ef598b76404b86e9350e120b86c9)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:04.585978 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74a704-c572-45c2-bd9f-6c55deb16d01 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:04.682860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4a74a704-c572-45c2-bd9f-6c55deb16d01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a297ef598b76404b86e9350e120b86c9', 'request_id': 'req-4a74a704-c572-45c2-bd9f-6c55deb16d01'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:04.696060 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4a74a704-c572-45c2-bd9f-6c55deb16d01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a297ef598b76404b86e9350e120b86c9', 'request_id': 'req-4a74a704-c572-45c2-bd9f-6c55deb16d01'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:04.708765 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4a74a704-c572-45c2-bd9f-6c55deb16d01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a297ef598b76404b86e9350e120b86c9', 'request_id': 'req-4a74a704-c572-45c2-bd9f-6c55deb16d01'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:04.736692 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4a74a704-c572-45c2-bd9f-6c55deb16d01 admin admin] The token cache is being invalidate because project a297ef598b76404b86e9350e120b86c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:04.737795 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4a74a704-c572-45c2-bd9f-6c55deb16d01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:04.740023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4a74a704-c572-45c2-bd9f-6c55deb16d01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:04.740438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1191/2406] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:04 2022] DELETE /identity/v3/projects/a297ef598b76404b86e9350e120b86c9 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:04.746730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc37f1-f22d-481d-9dc1-ea49e0900799 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:04.747015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc37f1-f22d-481d-9dc1-ea49e0900799 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:04.747321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc37f1-f22d-481d-9dc1-ea49e0900799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:04.774471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-80dc37f1-f22d-481d-9dc1-ea49e0900799 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:04.776204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-80dc37f1-f22d-481d-9dc1-ea49e0900799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:04.879594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1216/2407] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:04.887965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881687f2-33bd-4c11-a9f8-4d5d614c6551 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:04.889026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881687f2-33bd-4c11-a9f8-4d5d614c6551 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:04.889026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881687f2-33bd-4c11-a9f8-4d5d614c6551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:04.912241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-881687f2-33bd-4c11-a9f8-4d5d614c6551 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:04.913435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-881687f2-33bd-4c11-a9f8-4d5d614c6551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:04.927232 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1192/2408] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:04.932623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc37f1-f22d-481d-9dc1-ea49e0900799 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:04.949709 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc37f1-f22d-481d-9dc1-ea49e0900799 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:04.954950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f318c8c-4f0e-4c9d-a453-eab6e7a9409c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:04.955646 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f318c8c-4f0e-4c9d-a453-eab6e7a9409c admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:04.955965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f318c8c-4f0e-4c9d-a453-eab6e7a9409c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:04.956269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f318c8c-4f0e-4c9d-a453-eab6e7a9409c admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:04.956939 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f318c8c-4f0e-4c9d-a453-eab6e7a9409c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:32:04.957281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f318c8c-4f0e-4c9d-a453-eab6e7a9409c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:04.958946 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f318c8c-4f0e-4c9d-a453-eab6e7a9409c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:04.973126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1217/2409] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:32:04.977642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881687f2-33bd-4c11-a9f8-4d5d614c6551 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:04.988144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881687f2-33bd-4c11-a9f8-4d5d614c6551 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:04.994522 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54088ca5-8f99-493e-bf47-2ad9682b39a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:04.995304 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54088ca5-8f99-493e-bf47-2ad9682b39a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:04.995504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54088ca5-8f99-493e-bf47-2ad9682b39a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:04.995884 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54088ca5-8f99-493e-bf47-2ad9682b39a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:04.996489 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54088ca5-8f99-493e-bf47-2ad9682b39a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:04.998448 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54088ca5-8f99-493e-bf47-2ad9682b39a6 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:05.010924 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:05.033806 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1193/2410] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:04 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:05.038364 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f318c8c-4f0e-4c9d-a453-eab6e7a9409c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:05.050714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f318c8c-4f0e-4c9d-a453-eab6e7a9409c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:05.116155 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb7412-ade5-470f-9603-515f8e2dd13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:05.116632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb7412-ade5-470f-9603-515f8e2dd13b admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:05.116878 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb7412-ade5-470f-9603-515f8e2dd13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:05.117219 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb7412-ade5-470f-9603-515f8e2dd13b admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:05.117849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb7412-ade5-470f-9603-515f8e2dd13b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:05.119797 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb7412-ade5-470f-9603-515f8e2dd13b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:05.143196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1218/2411] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:05 2022] POST /identity/v3/users => generated 283 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:05.147430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54088ca5-8f99-493e-bf47-2ad9682b39a6 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:05.158508 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54088ca5-8f99-493e-bf47-2ad9682b39a6 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:05.163299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b474867-9d9b-45b8-8b6e-a145dd0656ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:05.163850 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b474867-9d9b-45b8-8b6e-a145dd0656ab admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:05.164129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b474867-9d9b-45b8-8b6e-a145dd0656ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:05.164401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b474867-9d9b-45b8-8b6e-a145dd0656ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:05.165077 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b474867-9d9b-45b8-8b6e-a145dd0656ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:05.166689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b474867-9d9b-45b8-8b6e-a145dd0656ab admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:05.178763 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1194/2412] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:05.183494 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb7412-ade5-470f-9603-515f8e2dd13b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:05.194347 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb7412-ade5-470f-9603-515f8e2dd13b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:05.199247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5fd6b-44cf-4ce9-9ed6-34e35ff79392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:05.199878 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5fd6b-44cf-4ce9-9ed6-34e35ff79392 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:05.200157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5fd6b-44cf-4ce9-9ed6-34e35ff79392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:05.200424 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5fd6b-44cf-4ce9-9ed6-34e35ff79392 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:05.201084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5fd6b-44cf-4ce9-9ed6-34e35ff79392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:05.202677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5fd6b-44cf-4ce9-9ed6-34e35ff79392 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:05.213320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1219/2413] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:05.220131 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b474867-9d9b-45b8-8b6e-a145dd0656ab admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:05.231208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b474867-9d9b-45b8-8b6e-a145dd0656ab admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:05.238295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52813f8-4da6-4e4e-9048-6c90afdd8d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:05.238978 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52813f8-4da6-4e4e-9048-6c90afdd8d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:05.239259 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52813f8-4da6-4e4e-9048-6c90afdd8d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:05.239530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52813f8-4da6-4e4e-9048-6c90afdd8d1f admin admin] PATH_INFO: `/v3/projects/4da962f413bc432ba54d3beea8b39abf/users/ab6569d2dc3c4b4f849a0c27aad193d7/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:05.258146 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52813f8-4da6-4e4e-9048-6c90afdd8d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4da962f413bc432ba54d3beea8b39abf, user_id=ab6569d2dc3c4b4f849a0c27aad193d7, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:05.260839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52813f8-4da6-4e4e-9048-6c90afdd8d1f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:05.276199 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1195/2414] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:05 2022] PUT /identity/v3/projects/4da962f413bc432ba54d3beea8b39abf/users/ab6569d2dc3c4b4f849a0c27aad193d7/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:05.288161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d04b6b-02ae-46e3-a505-3e462bf8440b None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:05.288422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d04b6b-02ae-46e3-a505-3e462bf8440b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:05.288702 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d04b6b-02ae-46e3-a505-3e462bf8440b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:05.319795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-79d04b6b-02ae-46e3-a505-3e462bf8440b None None] MFA Rules not processed for user `ab6569d2dc3c4b4f849a0c27aad193d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:05.321030 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-79d04b6b-02ae-46e3-a505-3e462bf8440b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:05.326448 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:05.394696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1220/2415] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:05.406630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52813f8-4da6-4e4e-9048-6c90afdd8d1f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:05.408344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d52813f8-4da6-4e4e-9048-6c90afdd8d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:05.469222 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52813f8-4da6-4e4e-9048-6c90afdd8d1f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:05.473721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd7380-25f9-4d9a-8579-f87139165809 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:05.474404 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd7380-25f9-4d9a-8579-f87139165809 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:05.474727 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd7380-25f9-4d9a-8579-f87139165809 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:05.475041 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd7380-25f9-4d9a-8579-f87139165809 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:05.485307 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd7380-25f9-4d9a-8579-f87139165809 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:05.486851 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd7380-25f9-4d9a-8579-f87139165809 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:05.497691 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1196/2416] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:32:12.081713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac026e-3d27-4fd7-9071-ec720056782b None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:12.082647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac026e-3d27-4fd7-9071-ec720056782b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:12.082647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac026e-3d27-4fd7-9071-ec720056782b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:12.110536 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e8ac026e-3d27-4fd7-9071-ec720056782b None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:12.111960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e8ac026e-3d27-4fd7-9071-ec720056782b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:12.189271 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1221/2417] 10.222.0.88 () {58 vars in 1009 bytes} [Tue Aug 23 22:32:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:12.203072 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd7380-25f9-4d9a-8579-f87139165809 service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:12.204675 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-12cd7380-25f9-4d9a-8579-f87139165809 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:12.293109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd7380-25f9-4d9a-8579-f87139165809 service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:12.308204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96b2ba-eff1-47e2-a898-0601b1c69ea5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:12.309197 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96b2ba-eff1-47e2-a898-0601b1c69ea5 service swift] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:12.309545 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96b2ba-eff1-47e2-a898-0601b1c69ea5 service swift] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:12.309778 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96b2ba-eff1-47e2-a898-0601b1c69ea5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:12.321556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96b2ba-eff1-47e2-a898-0601b1c69ea5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:12.323219 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96b2ba-eff1-47e2-a898-0601b1c69ea5 service swift] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:12.333850 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1197/2418] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:32:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:32:15.210730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac026e-3d27-4fd7-9071-ec720056782b None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:15.286853 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac026e-3d27-4fd7-9071-ec720056782b None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:15.291405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615aeb85-17a8-4abb-abaa-3d010d8ff337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:15.292056 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615aeb85-17a8-4abb-abaa-3d010d8ff337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:15.292337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615aeb85-17a8-4abb-abaa-3d010d8ff337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:15.292617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615aeb85-17a8-4abb-abaa-3d010d8ff337 admin admin] PATH_INFO: `/v3/users/ab6569d2dc3c4b4f849a0c27aad193d7` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:15.293756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615aeb85-17a8-4abb-abaa-3d010d8ff337 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab6569d2dc3c4b4f849a0c27aad193d7)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:15.295690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615aeb85-17a8-4abb-abaa-3d010d8ff337 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:15.355319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-615aeb85-17a8-4abb-abaa-3d010d8ff337 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab6569d2dc3c4b4f849a0c27aad193d7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:15.356731 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:15.372399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-615aeb85-17a8-4abb-abaa-3d010d8ff337 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab6569d2dc3c4b4f849a0c27aad193d7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:15.381239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-615aeb85-17a8-4abb-abaa-3d010d8ff337 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab6569d2dc3c4b4f849a0c27aad193d7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:15.381894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-615aeb85-17a8-4abb-abaa-3d010d8ff337 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab6569d2dc3c4b4f849a0c27aad193d7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:15.395904 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-615aeb85-17a8-4abb-abaa-3d010d8ff337 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab6569d2dc3c4b4f849a0c27aad193d7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:15.406186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1222/2419] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:15 2022] DELETE /identity/v3/users/ab6569d2dc3c4b4f849a0c27aad193d7 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:15.409281 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0788c0-2c66-45d7-a589-bd3afee7cd74 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:15.409758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0788c0-2c66-45d7-a589-bd3afee7cd74 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:15.410227 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0788c0-2c66-45d7-a589-bd3afee7cd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:15.457268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-6a0788c0-2c66-45d7-a589-bd3afee7cd74 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:15.458409 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6a0788c0-2c66-45d7-a589-bd3afee7cd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:15.511174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:15.535752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1198/2420] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:15.545823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615aeb85-17a8-4abb-abaa-3d010d8ff337 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:15.547140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-615aeb85-17a8-4abb-abaa-3d010d8ff337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:15.617016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615aeb85-17a8-4abb-abaa-3d010d8ff337 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:15.623695 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa9415-e23a-44cb-95c8-2491ab94e12f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:15.625151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa9415-e23a-44cb-95c8-2491ab94e12f service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:15.625365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa9415-e23a-44cb-95c8-2491ab94e12f service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:15.626314 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa9415-e23a-44cb-95c8-2491ab94e12f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:15.639122 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa9415-e23a-44cb-95c8-2491ab94e12f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:15.640686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa9415-e23a-44cb-95c8-2491ab94e12f service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:15.657020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1223/2421] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:16.085192 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0788c0-2c66-45d7-a589-bd3afee7cd74 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:16.086868 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6a0788c0-2c66-45d7-a589-bd3afee7cd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:16.101667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0788c0-2c66-45d7-a589-bd3afee7cd74 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:16.132398 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65945c78-6a60-4887-8262-3ffa8ea46248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:16.135652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65945c78-6a60-4887-8262-3ffa8ea46248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:16.135987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65945c78-6a60-4887-8262-3ffa8ea46248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:16.136386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65945c78-6a60-4887-8262-3ffa8ea46248 admin admin] PATH_INFO: `/v3/projects/4da962f413bc432ba54d3beea8b39abf` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:16.140642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65945c78-6a60-4887-8262-3ffa8ea46248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4da962f413bc432ba54d3beea8b39abf)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:16.146162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65945c78-6a60-4887-8262-3ffa8ea46248 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:16.244405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-65945c78-6a60-4887-8262-3ffa8ea46248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4da962f413bc432ba54d3beea8b39abf', 'request_id': 'req-65945c78-6a60-4887-8262-3ffa8ea46248'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:16.252959 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-65945c78-6a60-4887-8262-3ffa8ea46248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4da962f413bc432ba54d3beea8b39abf', 'request_id': 'req-65945c78-6a60-4887-8262-3ffa8ea46248'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:16.260565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-65945c78-6a60-4887-8262-3ffa8ea46248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4da962f413bc432ba54d3beea8b39abf', 'request_id': 'req-65945c78-6a60-4887-8262-3ffa8ea46248'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:16.277792 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-65945c78-6a60-4887-8262-3ffa8ea46248 admin admin] The token cache is being invalidate because project 4da962f413bc432ba54d3beea8b39abf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:16.278182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-65945c78-6a60-4887-8262-3ffa8ea46248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:16.279105 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-65945c78-6a60-4887-8262-3ffa8ea46248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:16.284671 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1199/2422] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:16 2022] DELETE /identity/v3/projects/4da962f413bc432ba54d3beea8b39abf => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:16.287891 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6be587-4e7a-4d11-9723-f811b16df879 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:16.288414 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6be587-4e7a-4d11-9723-f811b16df879 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:16.289886 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6be587-4e7a-4d11-9723-f811b16df879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:16.363338 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ba6be587-4e7a-4d11-9723-f811b16df879 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:16.366089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ba6be587-4e7a-4d11-9723-f811b16df879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:16.460033 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1224/2423] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:16.463214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133a279-d675-4caf-b937-999ea40f993b None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:16.463788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133a279-d675-4caf-b937-999ea40f993b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:16.463788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133a279-d675-4caf-b937-999ea40f993b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:16.486762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-f133a279-d675-4caf-b937-999ea40f993b None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:16.490028 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f133a279-d675-4caf-b937-999ea40f993b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:16.501132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1200/2424] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:16.502902 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6be587-4e7a-4d11-9723-f811b16df879 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:16.514893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6be587-4e7a-4d11-9723-f811b16df879 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:16.521981 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508258f6-79a0-4388-aad2-db2e1339c0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:16.522860 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508258f6-79a0-4388-aad2-db2e1339c0ab admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:16.523196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508258f6-79a0-4388-aad2-db2e1339c0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:16.523634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508258f6-79a0-4388-aad2-db2e1339c0ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:16.524296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508258f6-79a0-4388-aad2-db2e1339c0ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:32:16.524678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508258f6-79a0-4388-aad2-db2e1339c0ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:16.526487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508258f6-79a0-4388-aad2-db2e1339c0ab admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:16.540794 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1225/2425] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:32:16.545557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133a279-d675-4caf-b937-999ea40f993b None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:16.556952 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133a279-d675-4caf-b937-999ea40f993b None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:16.562632 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82beb1-283f-491a-aefa-a23b86ab9255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:16.563486 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82beb1-283f-491a-aefa-a23b86ab9255 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:16.563749 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82beb1-283f-491a-aefa-a23b86ab9255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:16.564026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82beb1-283f-491a-aefa-a23b86ab9255 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:16.564680 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82beb1-283f-491a-aefa-a23b86ab9255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:16.566679 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82beb1-283f-491a-aefa-a23b86ab9255 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:16.603833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1201/2426] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:16 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:16.611173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508258f6-79a0-4388-aad2-db2e1339c0ab admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:16.625243 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508258f6-79a0-4388-aad2-db2e1339c0ab admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:16.662915 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:16.711356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18231d7f-6bb4-4fc0-931f-c10e21c048dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:16.712373 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18231d7f-6bb4-4fc0-931f-c10e21c048dd admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:16.712821 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18231d7f-6bb4-4fc0-931f-c10e21c048dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:16.713696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18231d7f-6bb4-4fc0-931f-c10e21c048dd admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:16.714844 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18231d7f-6bb4-4fc0-931f-c10e21c048dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:16.718902 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18231d7f-6bb4-4fc0-931f-c10e21c048dd admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:16.746796 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1226/2427] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:16 2022] POST /identity/v3/users => generated 283 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:16.752396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82beb1-283f-491a-aefa-a23b86ab9255 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:16.764717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82beb1-283f-491a-aefa-a23b86ab9255 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:16.770124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb550423-b264-4ea9-8a0e-d7a2c0924dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:16.770756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb550423-b264-4ea9-8a0e-d7a2c0924dca admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:16.771033 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb550423-b264-4ea9-8a0e-d7a2c0924dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:16.771309 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb550423-b264-4ea9-8a0e-d7a2c0924dca admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:16.771979 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb550423-b264-4ea9-8a0e-d7a2c0924dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:16.773935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb550423-b264-4ea9-8a0e-d7a2c0924dca admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:16.790110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1202/2428] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:16.802189 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18231d7f-6bb4-4fc0-931f-c10e21c048dd admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:16.815119 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18231d7f-6bb4-4fc0-931f-c10e21c048dd admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:16.820536 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28567343-3861-438d-9693-645e6989d37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:16.821183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28567343-3861-438d-9693-645e6989d37e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:16.821470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28567343-3861-438d-9693-645e6989d37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:16.821749 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28567343-3861-438d-9693-645e6989d37e admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:16.822429 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28567343-3861-438d-9693-645e6989d37e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:16.824054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28567343-3861-438d-9693-645e6989d37e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:16.839029 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1227/2429] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:16.841978 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb550423-b264-4ea9-8a0e-d7a2c0924dca admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:16.855153 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb550423-b264-4ea9-8a0e-d7a2c0924dca admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:16.862606 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2dfeb-bf84-4351-bce7-40ae97d8de55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:16.863951 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2dfeb-bf84-4351-bce7-40ae97d8de55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:16.864498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2dfeb-bf84-4351-bce7-40ae97d8de55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:16.864992 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2dfeb-bf84-4351-bce7-40ae97d8de55 admin admin] PATH_INFO: `/v3/projects/7eb7991f2d7d474da8da53c80953be4b/users/19926cd9adcf4c859c67d14bf09b2520/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:16.891883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2dfeb-bf84-4351-bce7-40ae97d8de55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eb7991f2d7d474da8da53c80953be4b, user_id=19926cd9adcf4c859c67d14bf09b2520, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:16.894795 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2dfeb-bf84-4351-bce7-40ae97d8de55 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:16.916638 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1203/2430] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:16 2022] PUT /identity/v3/projects/7eb7991f2d7d474da8da53c80953be4b/users/19926cd9adcf4c859c67d14bf09b2520/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:16.936357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdd0da-7e1f-474a-81c9-3f93916608c2 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:16.936838 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdd0da-7e1f-474a-81c9-3f93916608c2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:16.937412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdd0da-7e1f-474a-81c9-3f93916608c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:16.978091 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7ffdd0da-7e1f-474a-81c9-3f93916608c2 None None] MFA Rules not processed for user `19926cd9adcf4c859c67d14bf09b2520`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:16.979660 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7ffdd0da-7e1f-474a-81c9-3f93916608c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:17.092947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1228/2431] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:17.108351 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2dfeb-bf84-4351-bce7-40ae97d8de55 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:17.192863 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2dfeb-bf84-4351-bce7-40ae97d8de55 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:17.197987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab24d6-536e-4250-b475-24888162b4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:17.198618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab24d6-536e-4250-b475-24888162b4f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:17.198897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab24d6-536e-4250-b475-24888162b4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:17.199172 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab24d6-536e-4250-b475-24888162b4f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:17.199830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab24d6-536e-4250-b475-24888162b4f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:17.201823 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab24d6-536e-4250-b475-24888162b4f2 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:17.220210 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:17.262446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1204/2432] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:17 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:17.271344 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdd0da-7e1f-474a-81c9-3f93916608c2 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:17.295834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdd0da-7e1f-474a-81c9-3f93916608c2 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:17.412197 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb5ce6-0c3f-422a-b4fa-7a3d2ae64d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:17.413299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb5ce6-0c3f-422a-b4fa-7a3d2ae64d49 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:17.413961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb5ce6-0c3f-422a-b4fa-7a3d2ae64d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:17.414589 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb5ce6-0c3f-422a-b4fa-7a3d2ae64d49 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:17.415936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb5ce6-0c3f-422a-b4fa-7a3d2ae64d49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:17.420229 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb5ce6-0c3f-422a-b4fa-7a3d2ae64d49 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:17.478551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1229/2433] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:17 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:17.479885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab24d6-536e-4250-b475-24888162b4f2 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:17.492724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab24d6-536e-4250-b475-24888162b4f2 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:17.507912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c3b9f-203e-42ad-ad7d-4a43ba9113f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:17.509139 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c3b9f-203e-42ad-ad7d-4a43ba9113f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:17.509673 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c3b9f-203e-42ad-ad7d-4a43ba9113f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:17.510107 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c3b9f-203e-42ad-ad7d-4a43ba9113f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:17.511031 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c3b9f-203e-42ad-ad7d-4a43ba9113f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:17.512714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c3b9f-203e-42ad-ad7d-4a43ba9113f9 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:17.526641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1205/2434] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:17.534084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb5ce6-0c3f-422a-b4fa-7a3d2ae64d49 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:17.551690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb5ce6-0c3f-422a-b4fa-7a3d2ae64d49 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:17.558596 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e9d65-f0bf-4001-a030-b54d0ab809e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:17.559292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e9d65-f0bf-4001-a030-b54d0ab809e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:17.559574 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e9d65-f0bf-4001-a030-b54d0ab809e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:17.559841 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e9d65-f0bf-4001-a030-b54d0ab809e3 admin admin] PATH_INFO: `/v3/projects/4ccf92eda64b4a5a9acbd2b10f8f99b2/users/6b584c3ea02d4d1997f94b5e0aca807c/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:17.582265 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e9d65-f0bf-4001-a030-b54d0ab809e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ccf92eda64b4a5a9acbd2b10f8f99b2, user_id=6b584c3ea02d4d1997f94b5e0aca807c, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:17.586272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e9d65-f0bf-4001-a030-b54d0ab809e3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:17.609683 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1230/2435] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:17 2022] PUT /identity/v3/projects/4ccf92eda64b4a5a9acbd2b10f8f99b2/users/6b584c3ea02d4d1997f94b5e0aca807c/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:17.614372 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c3b9f-203e-42ad-ad7d-4a43ba9113f9 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:17.625260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c3b9f-203e-42ad-ad7d-4a43ba9113f9 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:17.692336 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d357a76-e436-4045-abd0-0312765b874b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:17.692962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d357a76-e436-4045-abd0-0312765b874b admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:17.693328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d357a76-e436-4045-abd0-0312765b874b admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:17.693554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d357a76-e436-4045-abd0-0312765b874b admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:17.694206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d357a76-e436-4045-abd0-0312765b874b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:17.695964 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d357a76-e436-4045-abd0-0312765b874b admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:17.709441 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1206/2436] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:17.713486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e9d65-f0bf-4001-a030-b54d0ab809e3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:17.727461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:17.732938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e9d65-f0bf-4001-a030-b54d0ab809e3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:17.744059 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc78f5-c1c7-4408-ac52-0f3bbf231394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:17.745351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc78f5-c1c7-4408-ac52-0f3bbf231394 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:17.745979 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc78f5-c1c7-4408-ac52-0f3bbf231394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:17.746601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc78f5-c1c7-4408-ac52-0f3bbf231394 admin admin] PATH_INFO: `/v3/projects/4ccf92eda64b4a5a9acbd2b10f8f99b2/users/6b584c3ea02d4d1997f94b5e0aca807c/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:17.750885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc78f5-c1c7-4408-ac52-0f3bbf231394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ccf92eda64b4a5a9acbd2b10f8f99b2, user_id=6b584c3ea02d4d1997f94b5e0aca807c, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:17.754011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc78f5-c1c7-4408-ac52-0f3bbf231394 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:17.771804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1231/2437] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:17 2022] PUT /identity/v3/projects/4ccf92eda64b4a5a9acbd2b10f8f99b2/users/6b584c3ea02d4d1997f94b5e0aca807c/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:17.784071 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf008a-d1d3-4195-9c9f-ef976a518c44 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:17.784540 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf008a-d1d3-4195-9c9f-ef976a518c44 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:17.784765 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf008a-d1d3-4195-9c9f-ef976a518c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:17.823720 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5aaf008a-d1d3-4195-9c9f-ef976a518c44 None None] MFA Rules not processed for user `6b584c3ea02d4d1997f94b5e0aca807c`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:17.825026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5aaf008a-d1d3-4195-9c9f-ef976a518c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:17.932840 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1207/2438] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:17.941434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc78f5-c1c7-4408-ac52-0f3bbf231394 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:17.943267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9bbc78f5-c1c7-4408-ac52-0f3bbf231394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:18.025706 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc78f5-c1c7-4408-ac52-0f3bbf231394 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:18.030289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5d97b-b557-490d-a004-1ce5a2d068ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:18.031132 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5d97b-b557-490d-a004-1ce5a2d068ef service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:18.032168 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5d97b-b557-490d-a004-1ce5a2d068ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:18.032464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5d97b-b557-490d-a004-1ce5a2d068ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:18.048246 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5d97b-b557-490d-a004-1ce5a2d068ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:18.051161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5d97b-b557-490d-a004-1ce5a2d068ef service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:18.068945 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1232/2439] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:18.166865 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf008a-d1d3-4195-9c9f-ef976a518c44 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:18.196264 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf008a-d1d3-4195-9c9f-ef976a518c44 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:18.207145 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab854c-32ac-4065-b245-e4ef801754d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:18.207748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab854c-32ac-4065-b245-e4ef801754d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:18.208480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab854c-32ac-4065-b245-e4ef801754d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:18.208480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab854c-32ac-4065-b245-e4ef801754d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:18.336539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab854c-32ac-4065-b245-e4ef801754d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:18.336539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab854c-32ac-4065-b245-e4ef801754d6 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:18.345880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:18.361919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1208/2440] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:21.445242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5d97b-b557-490d-a004-1ce5a2d068ef service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:21.528464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5d97b-b557-490d-a004-1ce5a2d068ef service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:21.534075 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b286a4c-4baf-490e-9d78-aa0ba2021e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:21.534722 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b286a4c-4baf-490e-9d78-aa0ba2021e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:21.535001 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b286a4c-4baf-490e-9d78-aa0ba2021e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:21.535275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b286a4c-4baf-490e-9d78-aa0ba2021e08 admin admin] PATH_INFO: `/v3/users/19926cd9adcf4c859c67d14bf09b2520` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:21.536483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b286a4c-4baf-490e-9d78-aa0ba2021e08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19926cd9adcf4c859c67d14bf09b2520)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:21.538572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b286a4c-4baf-490e-9d78-aa0ba2021e08 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:21.600807 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6b286a4c-4baf-490e-9d78-aa0ba2021e08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19926cd9adcf4c859c67d14bf09b2520'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:21.620814 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6b286a4c-4baf-490e-9d78-aa0ba2021e08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19926cd9adcf4c859c67d14bf09b2520'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:21.636057 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6b286a4c-4baf-490e-9d78-aa0ba2021e08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19926cd9adcf4c859c67d14bf09b2520'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:21.636967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6b286a4c-4baf-490e-9d78-aa0ba2021e08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19926cd9adcf4c859c67d14bf09b2520'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:21.649078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6b286a4c-4baf-490e-9d78-aa0ba2021e08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19926cd9adcf4c859c67d14bf09b2520'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:21.664132 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1233/2441] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:21 2022] DELETE /identity/v3/users/19926cd9adcf4c859c67d14bf09b2520 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:21.670976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6517e2f-3866-4278-839d-cac9cd98b14f None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:21.671876 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6517e2f-3866-4278-839d-cac9cd98b14f None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:21.671876 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6517e2f-3866-4278-839d-cac9cd98b14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:21.704686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-e6517e2f-3866-4278-839d-cac9cd98b14f None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:21.709439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e6517e2f-3866-4278-839d-cac9cd98b14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:21.817681 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1209/2442] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:21.825741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b286a4c-4baf-490e-9d78-aa0ba2021e08 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:21.827260 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6b286a4c-4baf-490e-9d78-aa0ba2021e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:21.847333 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:21.918130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b286a4c-4baf-490e-9d78-aa0ba2021e08 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:21.922968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdb204-bf74-41dd-bfda-67add2b8376f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:21.923614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdb204-bf74-41dd-bfda-67add2b8376f service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:21.923894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdb204-bf74-41dd-bfda-67add2b8376f service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:21.924226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdb204-bf74-41dd-bfda-67add2b8376f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:21.936133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdb204-bf74-41dd-bfda-67add2b8376f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:21.939014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdb204-bf74-41dd-bfda-67add2b8376f service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:21.961175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1234/2443] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:22.268987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6517e2f-3866-4278-839d-cac9cd98b14f None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:22.271111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e6517e2f-3866-4278-839d-cac9cd98b14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:22.286139 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6517e2f-3866-4278-839d-cac9cd98b14f None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:22.291380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d7f4b-538d-4b19-a7e1-00301461ed95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:22.292127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d7f4b-538d-4b19-a7e1-00301461ed95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:22.292478 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d7f4b-538d-4b19-a7e1-00301461ed95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:22.292755 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d7f4b-538d-4b19-a7e1-00301461ed95 admin admin] PATH_INFO: `/v3/projects/7eb7991f2d7d474da8da53c80953be4b` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:22.294186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d7f4b-538d-4b19-a7e1-00301461ed95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eb7991f2d7d474da8da53c80953be4b)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:22.296137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d7f4b-538d-4b19-a7e1-00301461ed95 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:22.363898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5e8d7f4b-538d-4b19-a7e1-00301461ed95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eb7991f2d7d474da8da53c80953be4b', 'request_id': 'req-5e8d7f4b-538d-4b19-a7e1-00301461ed95'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:22.373232 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5e8d7f4b-538d-4b19-a7e1-00301461ed95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eb7991f2d7d474da8da53c80953be4b', 'request_id': 'req-5e8d7f4b-538d-4b19-a7e1-00301461ed95'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:22.382405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5e8d7f4b-538d-4b19-a7e1-00301461ed95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eb7991f2d7d474da8da53c80953be4b', 'request_id': 'req-5e8d7f4b-538d-4b19-a7e1-00301461ed95'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:22.397093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5e8d7f4b-538d-4b19-a7e1-00301461ed95 admin admin] The token cache is being invalidate because project 7eb7991f2d7d474da8da53c80953be4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:22.397507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5e8d7f4b-538d-4b19-a7e1-00301461ed95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:22.398239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-5e8d7f4b-538d-4b19-a7e1-00301461ed95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:22.399813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1210/2444] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:22 2022] DELETE /identity/v3/projects/7eb7991f2d7d474da8da53c80953be4b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:22.405033 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdb204-bf74-41dd-bfda-67add2b8376f service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:22.416566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdb204-bf74-41dd-bfda-67add2b8376f service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:22.478229 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ea513-679b-42ef-a29d-0a463c8b8f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:22.478940 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ea513-679b-42ef-a29d-0a463c8b8f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:22.479049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ea513-679b-42ef-a29d-0a463c8b8f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:22.479326 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ea513-679b-42ef-a29d-0a463c8b8f92 admin admin] PATH_INFO: `/v3/users/6b584c3ea02d4d1997f94b5e0aca807c` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:22.480471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ea513-679b-42ef-a29d-0a463c8b8f92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b584c3ea02d4d1997f94b5e0aca807c)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:22.482575 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ea513-679b-42ef-a29d-0a463c8b8f92 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:22.535263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1d6ea513-679b-42ef-a29d-0a463c8b8f92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b584c3ea02d4d1997f94b5e0aca807c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:22.548814 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1d6ea513-679b-42ef-a29d-0a463c8b8f92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b584c3ea02d4d1997f94b5e0aca807c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:22.560023 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1d6ea513-679b-42ef-a29d-0a463c8b8f92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b584c3ea02d4d1997f94b5e0aca807c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:22.560759 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1d6ea513-679b-42ef-a29d-0a463c8b8f92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b584c3ea02d4d1997f94b5e0aca807c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:22.578811 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1d6ea513-679b-42ef-a29d-0a463c8b8f92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b584c3ea02d4d1997f94b5e0aca807c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:22.585003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1235/2445] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:22 2022] DELETE /identity/v3/users/6b584c3ea02d4d1997f94b5e0aca807c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:22.883394 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d7f4b-538d-4b19-a7e1-00301461ed95 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:22.885404 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5e8d7f4b-538d-4b19-a7e1-00301461ed95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:22.900842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d7f4b-538d-4b19-a7e1-00301461ed95 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:22.973032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d35c9-b44b-4e3f-a9fd-17b28f470e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:22.973731 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d35c9-b44b-4e3f-a9fd-17b28f470e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:22.974018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d35c9-b44b-4e3f-a9fd-17b28f470e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:22.974291 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d35c9-b44b-4e3f-a9fd-17b28f470e04 admin admin] PATH_INFO: `/v3/projects/4ccf92eda64b4a5a9acbd2b10f8f99b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:22.975480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d35c9-b44b-4e3f-a9fd-17b28f470e04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ccf92eda64b4a5a9acbd2b10f8f99b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:22.977443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d35c9-b44b-4e3f-a9fd-17b28f470e04 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:23.005367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:23.040912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-545d35c9-b44b-4e3f-a9fd-17b28f470e04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ccf92eda64b4a5a9acbd2b10f8f99b2', 'request_id': 'req-545d35c9-b44b-4e3f-a9fd-17b28f470e04'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:23.047846 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-545d35c9-b44b-4e3f-a9fd-17b28f470e04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ccf92eda64b4a5a9acbd2b10f8f99b2', 'request_id': 'req-545d35c9-b44b-4e3f-a9fd-17b28f470e04'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:23.056270 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-545d35c9-b44b-4e3f-a9fd-17b28f470e04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ccf92eda64b4a5a9acbd2b10f8f99b2', 'request_id': 'req-545d35c9-b44b-4e3f-a9fd-17b28f470e04'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:23.073498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-545d35c9-b44b-4e3f-a9fd-17b28f470e04 admin admin] The token cache is being invalidate because project 4ccf92eda64b4a5a9acbd2b10f8f99b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:23.073830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-545d35c9-b44b-4e3f-a9fd-17b28f470e04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:23.074496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-545d35c9-b44b-4e3f-a9fd-17b28f470e04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:23.076033 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1211/2446] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:22 2022] DELETE /identity/v3/projects/4ccf92eda64b4a5a9acbd2b10f8f99b2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:23.082020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112adb45-30cc-4ab7-8d76-2805648f377f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:23.082288 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112adb45-30cc-4ab7-8d76-2805648f377f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:23.082616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112adb45-30cc-4ab7-8d76-2805648f377f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:23.107730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-112adb45-30cc-4ab7-8d76-2805648f377f None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:23.108999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-112adb45-30cc-4ab7-8d76-2805648f377f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:23.147289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:23.202161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1236/2447] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:23.210846 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb60d5-1330-467a-b56c-3027d3d52e74 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:23.211133 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb60d5-1330-467a-b56c-3027d3d52e74 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:23.211586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb60d5-1330-467a-b56c-3027d3d52e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:23.236277 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5edb60d5-1330-467a-b56c-3027d3d52e74 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:23.237432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5edb60d5-1330-467a-b56c-3027d3d52e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:23.247825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1212/2448] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:23.254649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112adb45-30cc-4ab7-8d76-2805648f377f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:23.270069 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112adb45-30cc-4ab7-8d76-2805648f377f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:23.276662 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f07f3-4746-4062-8456-3ef6976af038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:23.277328 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f07f3-4746-4062-8456-3ef6976af038 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:23.277607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f07f3-4746-4062-8456-3ef6976af038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:23.277967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f07f3-4746-4062-8456-3ef6976af038 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:23.278535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f07f3-4746-4062-8456-3ef6976af038 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:32:23.278840 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f07f3-4746-4062-8456-3ef6976af038 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:23.280535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f07f3-4746-4062-8456-3ef6976af038 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:23.294423 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1237/2449] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:32:23.300135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb60d5-1330-467a-b56c-3027d3d52e74 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:23.310816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb60d5-1330-467a-b56c-3027d3d52e74 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:23.316880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92632a-d606-4a8e-9ba8-34d75ecd27b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:23.317640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92632a-d606-4a8e-9ba8-34d75ecd27b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:23.317935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92632a-d606-4a8e-9ba8-34d75ecd27b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:23.318077 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92632a-d606-4a8e-9ba8-34d75ecd27b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:23.318736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92632a-d606-4a8e-9ba8-34d75ecd27b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:23.320736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92632a-d606-4a8e-9ba8-34d75ecd27b8 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:23.358723 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1213/2450] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:23 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:23.365198 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f07f3-4746-4062-8456-3ef6976af038 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:23.377311 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f07f3-4746-4062-8456-3ef6976af038 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:23.456088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55f65d-2d7a-4138-95bc-d4b41113335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:23.456730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55f65d-2d7a-4138-95bc-d4b41113335e admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:23.457015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55f65d-2d7a-4138-95bc-d4b41113335e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:23.457405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55f65d-2d7a-4138-95bc-d4b41113335e admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:23.458049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55f65d-2d7a-4138-95bc-d4b41113335e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:23.460053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55f65d-2d7a-4138-95bc-d4b41113335e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:23.485048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1238/2451] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:23 2022] POST /identity/v3/users => generated 285 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:23.489617 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92632a-d606-4a8e-9ba8-34d75ecd27b8 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:23.508896 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92632a-d606-4a8e-9ba8-34d75ecd27b8 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:23.513892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78fced-aca6-4d61-ade6-b44fcaef1f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:23.514528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78fced-aca6-4d61-ade6-b44fcaef1f8b admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:23.514813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78fced-aca6-4d61-ade6-b44fcaef1f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:23.515127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78fced-aca6-4d61-ade6-b44fcaef1f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:23.516324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78fced-aca6-4d61-ade6-b44fcaef1f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:23.518175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78fced-aca6-4d61-ade6-b44fcaef1f8b admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:23.531450 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1214/2452] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:23.536688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55f65d-2d7a-4138-95bc-d4b41113335e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:23.547505 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55f65d-2d7a-4138-95bc-d4b41113335e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:23.554392 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416245b9-81bc-47a5-b91d-588c710f4674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:23.555038 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416245b9-81bc-47a5-b91d-588c710f4674 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:23.555456 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416245b9-81bc-47a5-b91d-588c710f4674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:23.555746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416245b9-81bc-47a5-b91d-588c710f4674 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:23.556561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416245b9-81bc-47a5-b91d-588c710f4674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:23.558433 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416245b9-81bc-47a5-b91d-588c710f4674 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:23.574600 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1239/2453] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:23.576228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78fced-aca6-4d61-ade6-b44fcaef1f8b admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:23.589286 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78fced-aca6-4d61-ade6-b44fcaef1f8b admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:23.595740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0404b11-8283-4734-8eb5-f93ae31e8ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:23.596456 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0404b11-8283-4734-8eb5-f93ae31e8ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:23.596970 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0404b11-8283-4734-8eb5-f93ae31e8ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:23.597280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0404b11-8283-4734-8eb5-f93ae31e8ffe admin admin] PATH_INFO: `/v3/projects/bcd8882f30d4427c8fea75bf369b3337/users/2fb18eefd61446d7b5d9de853d304eb6/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:23.616449 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0404b11-8283-4734-8eb5-f93ae31e8ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcd8882f30d4427c8fea75bf369b3337, user_id=2fb18eefd61446d7b5d9de853d304eb6, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:23.619242 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0404b11-8283-4734-8eb5-f93ae31e8ffe admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:23.640226 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1215/2454] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:23 2022] PUT /identity/v3/projects/bcd8882f30d4427c8fea75bf369b3337/users/2fb18eefd61446d7b5d9de853d304eb6/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:23.658645 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8a161-8369-408a-ad24-6d147e4bf3a0 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:23.659024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8a161-8369-408a-ad24-6d147e4bf3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:23.659226 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8a161-8369-408a-ad24-6d147e4bf3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:23.694834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9ce8a161-8369-408a-ad24-6d147e4bf3a0 None None] MFA Rules not processed for user `2fb18eefd61446d7b5d9de853d304eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:23.696130 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9ce8a161-8369-408a-ad24-6d147e4bf3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:23.796064 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1240/2455] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:23.801515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0404b11-8283-4734-8eb5-f93ae31e8ffe admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:23.879766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0404b11-8283-4734-8eb5-f93ae31e8ffe admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:23.884911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c491cf-0957-4270-954f-0f6ccea7af2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:23.885688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c491cf-0957-4270-954f-0f6ccea7af2c admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:23.885974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c491cf-0957-4270-954f-0f6ccea7af2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:23.886247 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c491cf-0957-4270-954f-0f6ccea7af2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:23.886878 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c491cf-0957-4270-954f-0f6ccea7af2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:23.888788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c491cf-0957-4270-954f-0f6ccea7af2c admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:23.919768 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1216/2456] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:23 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:23.925206 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8a161-8369-408a-ad24-6d147e4bf3a0 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:23.936992 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8a161-8369-408a-ad24-6d147e4bf3a0 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:23.970003 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:24.008246 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0026458-17db-410a-b260-03f22fa10490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:24.009039 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0026458-17db-410a-b260-03f22fa10490 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:24.009357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0026458-17db-410a-b260-03f22fa10490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:24.009630 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0026458-17db-410a-b260-03f22fa10490 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:24.010266 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0026458-17db-410a-b260-03f22fa10490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:24.012232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0026458-17db-410a-b260-03f22fa10490 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:24.039535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1241/2457] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:23 2022] POST /identity/v3/users => generated 291 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:24.040662 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c491cf-0957-4270-954f-0f6ccea7af2c admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:24.053893 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c491cf-0957-4270-954f-0f6ccea7af2c admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:24.059029 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b50f43-dbe2-4e87-b8de-fa9ac86a9117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:24.059831 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b50f43-dbe2-4e87-b8de-fa9ac86a9117 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:24.060170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b50f43-dbe2-4e87-b8de-fa9ac86a9117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:24.060502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b50f43-dbe2-4e87-b8de-fa9ac86a9117 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:24.061394 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b50f43-dbe2-4e87-b8de-fa9ac86a9117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:24.063341 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b50f43-dbe2-4e87-b8de-fa9ac86a9117 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:24.079015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1217/2458] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:32:24.083473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0026458-17db-410a-b260-03f22fa10490 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:24.095257 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0026458-17db-410a-b260-03f22fa10490 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:24.100059 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182203c-34e6-46d7-8943-6f460406d1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:24.101394 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182203c-34e6-46d7-8943-6f460406d1d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:24.101394 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182203c-34e6-46d7-8943-6f460406d1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:24.101394 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182203c-34e6-46d7-8943-6f460406d1d9 admin admin] PATH_INFO: `/v3/projects/9de3a909ca044c87acc4b22c64b972cb/users/33adb878430f4199bcbc0f0154de7fc9/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:24.130018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182203c-34e6-46d7-8943-6f460406d1d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9de3a909ca044c87acc4b22c64b972cb, user_id=33adb878430f4199bcbc0f0154de7fc9, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:24.133389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182203c-34e6-46d7-8943-6f460406d1d9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:24.151093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1242/2459] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:24 2022] PUT /identity/v3/projects/9de3a909ca044c87acc4b22c64b972cb/users/33adb878430f4199bcbc0f0154de7fc9/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:24.157971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b50f43-dbe2-4e87-b8de-fa9ac86a9117 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:24.169011 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b50f43-dbe2-4e87-b8de-fa9ac86a9117 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:24.192108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:24.254254 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11efda7-dca9-4d86-adc1-6af819c0b37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:24.254978 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11efda7-dca9-4d86-adc1-6af819c0b37d admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:24.255186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11efda7-dca9-4d86-adc1-6af819c0b37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:24.255417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11efda7-dca9-4d86-adc1-6af819c0b37d admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:24.256123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11efda7-dca9-4d86-adc1-6af819c0b37d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:24.257890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11efda7-dca9-4d86-adc1-6af819c0b37d admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:24.271991 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1218/2460] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:24.276508 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182203c-34e6-46d7-8943-6f460406d1d9 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:24.288786 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182203c-34e6-46d7-8943-6f460406d1d9 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:24.295538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079e17d-fc86-466f-8944-c70c988b9b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:24.296356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079e17d-fc86-466f-8944-c70c988b9b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:24.296815 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079e17d-fc86-466f-8944-c70c988b9b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:24.297291 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079e17d-fc86-466f-8944-c70c988b9b2a admin admin] PATH_INFO: `/v3/projects/9de3a909ca044c87acc4b22c64b972cb/users/33adb878430f4199bcbc0f0154de7fc9/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:24.299566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079e17d-fc86-466f-8944-c70c988b9b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9de3a909ca044c87acc4b22c64b972cb, user_id=33adb878430f4199bcbc0f0154de7fc9, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:24.302929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079e17d-fc86-466f-8944-c70c988b9b2a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:24.315202 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1243/2461] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:24 2022] PUT /identity/v3/projects/9de3a909ca044c87acc4b22c64b972cb/users/33adb878430f4199bcbc0f0154de7fc9/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:24.327148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c59c89-32cc-48fc-b7c1-6d35d60a0b02 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:24.327520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c59c89-32cc-48fc-b7c1-6d35d60a0b02 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:24.327862 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c59c89-32cc-48fc-b7c1-6d35d60a0b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:24.361008 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-c3c59c89-32cc-48fc-b7c1-6d35d60a0b02 None None] MFA Rules not processed for user `33adb878430f4199bcbc0f0154de7fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:24.363054 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c3c59c89-32cc-48fc-b7c1-6d35d60a0b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:24.448044 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1219/2462] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:24.460872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079e17d-fc86-466f-8944-c70c988b9b2a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:24.462849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9079e17d-fc86-466f-8944-c70c988b9b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:24.537889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079e17d-fc86-466f-8944-c70c988b9b2a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:24.542738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cc557-3447-4052-a373-0a0a4d87ad46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:24.543492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cc557-3447-4052-a373-0a0a4d87ad46 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:24.544027 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cc557-3447-4052-a373-0a0a4d87ad46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:24.544372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cc557-3447-4052-a373-0a0a4d87ad46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:24.562104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cc557-3447-4052-a373-0a0a4d87ad46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:24.564178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cc557-3447-4052-a373-0a0a4d87ad46 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:24.578021 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1244/2463] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:26.575811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c59c89-32cc-48fc-b7c1-6d35d60a0b02 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:26.721126 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c59c89-32cc-48fc-b7c1-6d35d60a0b02 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:26.729646 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146bb4f-f194-4db0-b154-914fb79b5924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:26.730738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146bb4f-f194-4db0-b154-914fb79b5924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:26.731259 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146bb4f-f194-4db0-b154-914fb79b5924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:26.731681 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146bb4f-f194-4db0-b154-914fb79b5924 admin admin] PATH_INFO: `/v3/users/2fb18eefd61446d7b5d9de853d304eb6` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:26.733595 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146bb4f-f194-4db0-b154-914fb79b5924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fb18eefd61446d7b5d9de853d304eb6)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:26.736806 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146bb4f-f194-4db0-b154-914fb79b5924 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:26.812354 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:26.816355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-a146bb4f-f194-4db0-b154-914fb79b5924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fb18eefd61446d7b5d9de853d304eb6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:26.816735 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-a146bb4f-f194-4db0-b154-914fb79b5924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fb18eefd61446d7b5d9de853d304eb6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:26.830519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-a146bb4f-f194-4db0-b154-914fb79b5924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fb18eefd61446d7b5d9de853d304eb6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:26.848816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-a146bb4f-f194-4db0-b154-914fb79b5924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fb18eefd61446d7b5d9de853d304eb6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:26.862400 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-a146bb4f-f194-4db0-b154-914fb79b5924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fb18eefd61446d7b5d9de853d304eb6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:26.875341 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1220/2464] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:26 2022] DELETE /identity/v3/users/2fb18eefd61446d7b5d9de853d304eb6 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:26.883205 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1638f-3807-43a2-8805-8222ee5ca2a2 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:26.885447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1638f-3807-43a2-8805-8222ee5ca2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:26.885755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1638f-3807-43a2-8805-8222ee5ca2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:26.914701 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9ac1638f-3807-43a2-8805-8222ee5ca2a2 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:26.917318 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9ac1638f-3807-43a2-8805-8222ee5ca2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:27.014580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1245/2465] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:27.026851 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146bb4f-f194-4db0-b154-914fb79b5924 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:27.028575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a146bb4f-f194-4db0-b154-914fb79b5924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:27.110680 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146bb4f-f194-4db0-b154-914fb79b5924 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:27.115598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e39d3-1d67-4339-86f3-95318a972f17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:27.116394 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e39d3-1d67-4339-86f3-95318a972f17 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:27.116574 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e39d3-1d67-4339-86f3-95318a972f17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:27.116772 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e39d3-1d67-4339-86f3-95318a972f17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:27.128047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e39d3-1d67-4339-86f3-95318a972f17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:27.129545 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e39d3-1d67-4339-86f3-95318a972f17 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:27.140395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1221/2466] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:27.500109 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1638f-3807-43a2-8805-8222ee5ca2a2 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:27.502126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9ac1638f-3807-43a2-8805-8222ee5ca2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:27.519675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1638f-3807-43a2-8805-8222ee5ca2a2 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:27.524967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:27.525670 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:27.525961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:27.526434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8 admin admin] PATH_INFO: `/v3/projects/bcd8882f30d4427c8fea75bf369b3337` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:27.527673 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcd8882f30d4427c8fea75bf369b3337)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:27.529659 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:27.575102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:27.607400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcd8882f30d4427c8fea75bf369b3337', 'request_id': 'req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:27.616017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcd8882f30d4427c8fea75bf369b3337', 'request_id': 'req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:27.628753 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcd8882f30d4427c8fea75bf369b3337', 'request_id': 'req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:27.652022 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8 admin admin] The token cache is being invalidate because project bcd8882f30d4427c8fea75bf369b3337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:27.652458 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:27.653580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:27.657287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1246/2467] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:27 2022] DELETE /identity/v3/projects/bcd8882f30d4427c8fea75bf369b3337 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:27.661865 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e39d3-1d67-4339-86f3-95318a972f17 service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:27.674724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e39d3-1d67-4339-86f3-95318a972f17 service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:27.753135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef9f95-59a2-44db-8eac-0128c8d031c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:27.754004 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef9f95-59a2-44db-8eac-0128c8d031c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:27.754228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef9f95-59a2-44db-8eac-0128c8d031c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:27.759407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef9f95-59a2-44db-8eac-0128c8d031c2 admin admin] PATH_INFO: `/v3/users/33adb878430f4199bcbc0f0154de7fc9` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:27.759407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef9f95-59a2-44db-8eac-0128c8d031c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33adb878430f4199bcbc0f0154de7fc9)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:27.759407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef9f95-59a2-44db-8eac-0128c8d031c2 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:27.841795 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-64ef9f95-59a2-44db-8eac-0128c8d031c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33adb878430f4199bcbc0f0154de7fc9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:27.842485 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-64ef9f95-59a2-44db-8eac-0128c8d031c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33adb878430f4199bcbc0f0154de7fc9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:27.851211 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-64ef9f95-59a2-44db-8eac-0128c8d031c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33adb878430f4199bcbc0f0154de7fc9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:27.861682 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:27.876320 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-64ef9f95-59a2-44db-8eac-0128c8d031c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33adb878430f4199bcbc0f0154de7fc9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:27.886240 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-64ef9f95-59a2-44db-8eac-0128c8d031c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33adb878430f4199bcbc0f0154de7fc9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:27.893895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1222/2468] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:27 2022] DELETE /identity/v3/users/33adb878430f4199bcbc0f0154de7fc9 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:28.275987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:28.277630 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:28.293812 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4afe4-1fcf-4e59-982b-2cf67c70f8d8 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:28.358715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6d0b9-d9f0-4dca-97a6-f242e8299568 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:28.359229 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6d0b9-d9f0-4dca-97a6-f242e8299568 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:28.360011 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6d0b9-d9f0-4dca-97a6-f242e8299568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:28.394007 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:28.395027 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:28.395304 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:28.395776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1 admin admin] PATH_INFO: `/v3/projects/9de3a909ca044c87acc4b22c64b972cb` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:28.397529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9de3a909ca044c87acc4b22c64b972cb)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:28.399543 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:28.415600 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-14c6d0b9-d9f0-4dca-97a6-f242e8299568 None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:28.418167 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-14c6d0b9-d9f0-4dca-97a6-f242e8299568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:28.477387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9de3a909ca044c87acc4b22c64b972cb', 'request_id': 'req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:28.487720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9de3a909ca044c87acc4b22c64b972cb', 'request_id': 'req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:28.499454 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9de3a909ca044c87acc4b22c64b972cb', 'request_id': 'req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:28.507209 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1223/2469] 10.222.0.88 () {58 vars in 967 bytes} [Tue Aug 23 22:32:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:28.516292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1 admin admin] The token cache is being invalidate because project 9de3a909ca044c87acc4b22c64b972cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:28.516616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:28.518400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:28.520068 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1247/2470] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:28 2022] DELETE /identity/v3/projects/9de3a909ca044c87acc4b22c64b972cb => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:28.525427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb11606-3346-439c-8945-515f315b7c4c None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:28.525741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb11606-3346-439c-8945-515f315b7c4c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:28.526027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb11606-3346-439c-8945-515f315b7c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:28.526807 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:28.528360 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:28.555034 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-ffb11606-3346-439c-8945-515f315b7c4c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:28.556725 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ffb11606-3346-439c-8945-515f315b7c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:28.600635 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:28.652783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0a9ac-76d3-43e9-86f2-1bd6a9c420c1 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:28.655108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1224/2471] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:28.661551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab9733-7aa9-454a-aa8e-54a4d04b8c0a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:28.662714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab9733-7aa9-454a-aa8e-54a4d04b8c0a service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:28.663204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab9733-7aa9-454a-aa8e-54a4d04b8c0a service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:28.663679 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab9733-7aa9-454a-aa8e-54a4d04b8c0a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:28.665543 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf843dd6-ef18-4135-bf75-9a91535f3b98 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:28.666049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf843dd6-ef18-4135-bf75-9a91535f3b98 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:28.666219 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-87ab9733-7aa9-454a-aa8e-54a4d04b8c0a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:28.666384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf843dd6-ef18-4135-bf75-9a91535f3b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:28.690954 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-bf843dd6-ef18-4135-bf75-9a91535f3b98 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:28.692297 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-bf843dd6-ef18-4135-bf75-9a91535f3b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:28.700802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1225/2472] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:28.707091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf843dd6-ef18-4135-bf75-9a91535f3b98 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:28.720971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf843dd6-ef18-4135-bf75-9a91535f3b98 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:28.730375 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210ba16-4343-446e-a467-1eb1da233f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:28.730892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210ba16-4343-446e-a467-1eb1da233f52 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:28.731150 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210ba16-4343-446e-a467-1eb1da233f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:28.731425 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210ba16-4343-446e-a467-1eb1da233f52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:28.732266 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210ba16-4343-446e-a467-1eb1da233f52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:32:28.732615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210ba16-4343-446e-a467-1eb1da233f52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:28.734948 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210ba16-4343-446e-a467-1eb1da233f52 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:28.748812 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1226/2473] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:32:28.754067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210ba16-4343-446e-a467-1eb1da233f52 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:28.766356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210ba16-4343-446e-a467-1eb1da233f52 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:28.772087 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f403c1-5da7-470d-9618-46e5f92d6025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:28.772811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f403c1-5da7-470d-9618-46e5f92d6025 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:28.773715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f403c1-5da7-470d-9618-46e5f92d6025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:28.774214 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f403c1-5da7-470d-9618-46e5f92d6025 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:28.775380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f403c1-5da7-470d-9618-46e5f92d6025 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:28.778222 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f403c1-5da7-470d-9618-46e5f92d6025 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:28.782134 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab9733-7aa9-454a-aa8e-54a4d04b8c0a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:28.784745 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab9733-7aa9-454a-aa8e-54a4d04b8c0a service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:28.815485 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1248/2474] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:32:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:32:28.861488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1227/2475] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:28 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:28.863339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab9733-7aa9-454a-aa8e-54a4d04b8c0a service swift] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:28.884637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab9733-7aa9-454a-aa8e-54a4d04b8c0a service swift] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:28.997337 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fed89-6e08-42eb-94d6-abe6582140da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:28.998407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fed89-6e08-42eb-94d6-abe6582140da admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:28.998697 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fed89-6e08-42eb-94d6-abe6582140da admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:28.999083 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fed89-6e08-42eb-94d6-abe6582140da admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:29.000684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fed89-6e08-42eb-94d6-abe6582140da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:29.004397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fed89-6e08-42eb-94d6-abe6582140da admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:29.041984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1249/2476] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:28 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:29.045169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f403c1-5da7-470d-9618-46e5f92d6025 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:29.070207 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f403c1-5da7-470d-9618-46e5f92d6025 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:29.076150 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650de3bd-1434-46e8-8707-549b2d7a6230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:29.076802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650de3bd-1434-46e8-8707-549b2d7a6230 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:29.077229 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650de3bd-1434-46e8-8707-549b2d7a6230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:29.077607 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650de3bd-1434-46e8-8707-549b2d7a6230 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:29.078861 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650de3bd-1434-46e8-8707-549b2d7a6230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:29.084091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650de3bd-1434-46e8-8707-549b2d7a6230 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:29.098726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1228/2477] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:29.105172 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fed89-6e08-42eb-94d6-abe6582140da admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:29.123405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fed89-6e08-42eb-94d6-abe6582140da admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:29.132121 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257734a-15d6-487b-861f-2cff63bad07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:29.133106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257734a-15d6-487b-861f-2cff63bad07d admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:29.133428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257734a-15d6-487b-861f-2cff63bad07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:29.133869 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257734a-15d6-487b-861f-2cff63bad07d admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:29.134679 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257734a-15d6-487b-861f-2cff63bad07d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:29.136502 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257734a-15d6-487b-861f-2cff63bad07d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:29.159786 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1250/2478] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:29.164410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650de3bd-1434-46e8-8707-549b2d7a6230 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:29.186974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650de3bd-1434-46e8-8707-549b2d7a6230 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:29.195662 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d512b4-d1b5-4ff6-8a8a-3a0a73e1dc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:29.196973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d512b4-d1b5-4ff6-8a8a-3a0a73e1dc50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:29.197657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d512b4-d1b5-4ff6-8a8a-3a0a73e1dc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:29.198160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d512b4-d1b5-4ff6-8a8a-3a0a73e1dc50 admin admin] PATH_INFO: `/v3/projects/b818196b96b84243b04799802013f636/users/e613daa6d40646a1bd045d3a6b211969/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:29.226706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d512b4-d1b5-4ff6-8a8a-3a0a73e1dc50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b818196b96b84243b04799802013f636, user_id=e613daa6d40646a1bd045d3a6b211969, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:29.229873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d512b4-d1b5-4ff6-8a8a-3a0a73e1dc50 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:29.246440 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1229/2479] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:29 2022] PUT /identity/v3/projects/b818196b96b84243b04799802013f636/users/e613daa6d40646a1bd045d3a6b211969/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:29.259590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8446dae-0349-454c-b9c3-26db6b9c12b3 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:29.259914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8446dae-0349-454c-b9c3-26db6b9c12b3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:29.260070 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8446dae-0349-454c-b9c3-26db6b9c12b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:29.296161 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d8446dae-0349-454c-b9c3-26db6b9c12b3 None None] MFA Rules not processed for user `e613daa6d40646a1bd045d3a6b211969`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:29.298072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d8446dae-0349-454c-b9c3-26db6b9c12b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:29.362180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:29.386691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1251/2480] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:29.393587 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d512b4-d1b5-4ff6-8a8a-3a0a73e1dc50 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:29.399983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:29.482824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d512b4-d1b5-4ff6-8a8a-3a0a73e1dc50 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:29.489027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e387f-977e-4165-b624-effddbe42485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:29.489789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e387f-977e-4165-b624-effddbe42485 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:29.490305 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e387f-977e-4165-b624-effddbe42485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:29.490598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e387f-977e-4165-b624-effddbe42485 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:29.491178 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e387f-977e-4165-b624-effddbe42485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:29.493837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e387f-977e-4165-b624-effddbe42485 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:29.526801 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1230/2481] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:29 2022] POST /identity/v3/projects => generated 367 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:29.532779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8446dae-0349-454c-b9c3-26db6b9c12b3 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:29.545174 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8446dae-0349-454c-b9c3-26db6b9c12b3 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:29.623076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0d04d-9176-4a77-887b-c1ee6f3c181f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:29.623708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0d04d-9176-4a77-887b-c1ee6f3c181f admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:29.623987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0d04d-9176-4a77-887b-c1ee6f3c181f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:29.624255 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0d04d-9176-4a77-887b-c1ee6f3c181f admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:29.624906 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0d04d-9176-4a77-887b-c1ee6f3c181f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:29.627124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0d04d-9176-4a77-887b-c1ee6f3c181f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:29.656254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1252/2482] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:29 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:29.662437 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e387f-977e-4165-b624-effddbe42485 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:29.674504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e387f-977e-4165-b624-effddbe42485 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:29.681211 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df3729-d026-4a95-832e-e2074ebca7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:29.681862 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df3729-d026-4a95-832e-e2074ebca7cd admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:29.682165 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df3729-d026-4a95-832e-e2074ebca7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:29.682416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df3729-d026-4a95-832e-e2074ebca7cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:29.683078 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df3729-d026-4a95-832e-e2074ebca7cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:29.684714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df3729-d026-4a95-832e-e2074ebca7cd admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:29.698010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1231/2483] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:29.702894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0d04d-9176-4a77-887b-c1ee6f3c181f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:29.714015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0d04d-9176-4a77-887b-c1ee6f3c181f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:29.721391 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ae80c-0cfd-417f-b73f-05e9c79a1da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:29.722197 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae80c-0cfd-417f-b73f-05e9c79a1da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:29.722591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae80c-0cfd-417f-b73f-05e9c79a1da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:29.722974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae80c-0cfd-417f-b73f-05e9c79a1da2 admin admin] PATH_INFO: `/v3/projects/1b7617dac3914c90b31955da1108babf/users/87a971d9b65d4b298fa97db00ad8b393/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:29.749601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ae80c-0cfd-417f-b73f-05e9c79a1da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b7617dac3914c90b31955da1108babf, user_id=87a971d9b65d4b298fa97db00ad8b393, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:29.755601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ae80c-0cfd-417f-b73f-05e9c79a1da2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:29.777468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1253/2484] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:29 2022] PUT /identity/v3/projects/1b7617dac3914c90b31955da1108babf/users/87a971d9b65d4b298fa97db00ad8b393/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:29.781493 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df3729-d026-4a95-832e-e2074ebca7cd admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:29.793565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df3729-d026-4a95-832e-e2074ebca7cd admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:29.865686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80317bd3-9d2d-42e1-8dda-7fff98772f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:29.867023 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80317bd3-9d2d-42e1-8dda-7fff98772f62 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:29.867305 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80317bd3-9d2d-42e1-8dda-7fff98772f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:29.867579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80317bd3-9d2d-42e1-8dda-7fff98772f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:29.868546 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80317bd3-9d2d-42e1-8dda-7fff98772f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:29.870089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80317bd3-9d2d-42e1-8dda-7fff98772f62 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:29.884059 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1232/2485] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:29.888972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ae80c-0cfd-417f-b73f-05e9c79a1da2 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:29.902880 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ae80c-0cfd-417f-b73f-05e9c79a1da2 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:29.909397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459edf00-17c9-4db1-a43d-5beb62a5e9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:29.910250 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459edf00-17c9-4db1-a43d-5beb62a5e9d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:29.910610 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459edf00-17c9-4db1-a43d-5beb62a5e9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:29.911050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459edf00-17c9-4db1-a43d-5beb62a5e9d8 admin admin] PATH_INFO: `/v3/projects/1b7617dac3914c90b31955da1108babf/users/87a971d9b65d4b298fa97db00ad8b393/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:29.913324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459edf00-17c9-4db1-a43d-5beb62a5e9d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b7617dac3914c90b31955da1108babf, user_id=87a971d9b65d4b298fa97db00ad8b393, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:29.916108 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459edf00-17c9-4db1-a43d-5beb62a5e9d8 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:29.929591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1254/2486] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:29 2022] PUT /identity/v3/projects/1b7617dac3914c90b31955da1108babf/users/87a971d9b65d4b298fa97db00ad8b393/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:29.941168 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460b7b0-97ca-4545-99ae-7feb7f2387aa None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:29.941925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460b7b0-97ca-4545-99ae-7feb7f2387aa None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:29.942720 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460b7b0-97ca-4545-99ae-7feb7f2387aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:29.981385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-3460b7b0-97ca-4545-99ae-7feb7f2387aa None None] MFA Rules not processed for user `87a971d9b65d4b298fa97db00ad8b393`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:29.982911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3460b7b0-97ca-4545-99ae-7feb7f2387aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:30.104774 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:30.116611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1233/2487] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:30.135593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459edf00-17c9-4db1-a43d-5beb62a5e9d8 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:30.137498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-459edf00-17c9-4db1-a43d-5beb62a5e9d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:30.228660 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459edf00-17c9-4db1-a43d-5beb62a5e9d8 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:30.234429 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70533f3e-9bc1-461d-94ea-c5ceaa9f5f05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:30.235537 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70533f3e-9bc1-461d-94ea-c5ceaa9f5f05 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:30.235975 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70533f3e-9bc1-461d-94ea-c5ceaa9f5f05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:30.237097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70533f3e-9bc1-461d-94ea-c5ceaa9f5f05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:30.251575 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70533f3e-9bc1-461d-94ea-c5ceaa9f5f05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:30.254784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70533f3e-9bc1-461d-94ea-c5ceaa9f5f05 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:30.275930 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1255/2488] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:30.346300 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460b7b0-97ca-4545-99ae-7feb7f2387aa None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:30.348431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3460b7b0-97ca-4545-99ae-7feb7f2387aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:30.365799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460b7b0-97ca-4545-99ae-7feb7f2387aa None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:30.379816 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0ea5c-5b99-43da-8d00-15e8de7929fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:30.380767 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0ea5c-5b99-43da-8d00-15e8de7929fe service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:30.381129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0ea5c-5b99-43da-8d00-15e8de7929fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:30.381415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0ea5c-5b99-43da-8d00-15e8de7929fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:30.462603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0ea5c-5b99-43da-8d00-15e8de7929fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:30.464092 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0ea5c-5b99-43da-8d00-15e8de7929fe service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:30.474604 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1234/2489] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:34.417528 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70533f3e-9bc1-461d-94ea-c5ceaa9f5f05 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:34.479894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:34.506826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70533f3e-9bc1-461d-94ea-c5ceaa9f5f05 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:34.513020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db55be-a6e2-49c5-9e20-401725549bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:34.513834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db55be-a6e2-49c5-9e20-401725549bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:34.514200 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db55be-a6e2-49c5-9e20-401725549bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:34.514588 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db55be-a6e2-49c5-9e20-401725549bbe admin admin] PATH_INFO: `/v3/users/e613daa6d40646a1bd045d3a6b211969` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:34.517278 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db55be-a6e2-49c5-9e20-401725549bbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=e613daa6d40646a1bd045d3a6b211969)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:34.519311 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db55be-a6e2-49c5-9e20-401725549bbe admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:34.595856 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e9db55be-a6e2-49c5-9e20-401725549bbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e613daa6d40646a1bd045d3a6b211969'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:34.615472 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e9db55be-a6e2-49c5-9e20-401725549bbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e613daa6d40646a1bd045d3a6b211969'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:34.626763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e9db55be-a6e2-49c5-9e20-401725549bbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e613daa6d40646a1bd045d3a6b211969'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:34.627654 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e9db55be-a6e2-49c5-9e20-401725549bbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e613daa6d40646a1bd045d3a6b211969'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:34.639741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e9db55be-a6e2-49c5-9e20-401725549bbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e613daa6d40646a1bd045d3a6b211969'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:34.648252 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1256/2490] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:34 2022] DELETE /identity/v3/users/e613daa6d40646a1bd045d3a6b211969 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:34.653225 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25198560-22e5-4b4d-8438-49a7bb833e65 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:34.653814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25198560-22e5-4b4d-8438-49a7bb833e65 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:34.653814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25198560-22e5-4b4d-8438-49a7bb833e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:34.687743 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-25198560-22e5-4b4d-8438-49a7bb833e65 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:34.689361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-25198560-22e5-4b4d-8438-49a7bb833e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:34.799309 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1235/2491] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:34.813239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db55be-a6e2-49c5-9e20-401725549bbe admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:34.815102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e9db55be-a6e2-49c5-9e20-401725549bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:34.910486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db55be-a6e2-49c5-9e20-401725549bbe admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:34.915317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a1597-021b-4aec-b54b-788222fdccdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:34.916009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a1597-021b-4aec-b54b-788222fdccdf service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:34.916297 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a1597-021b-4aec-b54b-788222fdccdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:34.916574 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a1597-021b-4aec-b54b-788222fdccdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:34.929088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a1597-021b-4aec-b54b-788222fdccdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:34.931866 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a1597-021b-4aec-b54b-788222fdccdf service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:34.956097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1257/2492] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:35.354748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25198560-22e5-4b4d-8438-49a7bb833e65 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:35.356473 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-25198560-22e5-4b4d-8438-49a7bb833e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:35.386482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25198560-22e5-4b4d-8438-49a7bb833e65 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:35.393929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:35.394602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:35.394885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:35.395197 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407 admin admin] PATH_INFO: `/v3/projects/b818196b96b84243b04799802013f636` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:35.396489 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b818196b96b84243b04799802013f636)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:35.398676 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:35.480332 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b818196b96b84243b04799802013f636', 'request_id': 'req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:35.494423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b818196b96b84243b04799802013f636', 'request_id': 'req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:35.503400 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b818196b96b84243b04799802013f636', 'request_id': 'req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:35.519586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407 admin admin] The token cache is being invalidate because project b818196b96b84243b04799802013f636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:35.519860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:35.520558 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:35.522135 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1236/2493] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:35 2022] DELETE /identity/v3/projects/b818196b96b84243b04799802013f636 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:35.527230 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a1597-021b-4aec-b54b-788222fdccdf service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:35.538720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a1597-021b-4aec-b54b-788222fdccdf service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:35.647539 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2e7d6-bc54-46b0-95f8-e796b5ea08a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:35.648643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2e7d6-bc54-46b0-95f8-e796b5ea08a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:35.649211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2e7d6-bc54-46b0-95f8-e796b5ea08a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:35.649696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2e7d6-bc54-46b0-95f8-e796b5ea08a0 admin admin] PATH_INFO: `/v3/users/87a971d9b65d4b298fa97db00ad8b393` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:35.651758 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2e7d6-bc54-46b0-95f8-e796b5ea08a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87a971d9b65d4b298fa97db00ad8b393)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:35.656827 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2e7d6-bc54-46b0-95f8-e796b5ea08a0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:35.666294 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:35.736215 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8ef2e7d6-bc54-46b0-95f8-e796b5ea08a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87a971d9b65d4b298fa97db00ad8b393'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:35.756636 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8ef2e7d6-bc54-46b0-95f8-e796b5ea08a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87a971d9b65d4b298fa97db00ad8b393'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:35.771116 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8ef2e7d6-bc54-46b0-95f8-e796b5ea08a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87a971d9b65d4b298fa97db00ad8b393'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:35.771997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8ef2e7d6-bc54-46b0-95f8-e796b5ea08a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87a971d9b65d4b298fa97db00ad8b393'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:35.784714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8ef2e7d6-bc54-46b0-95f8-e796b5ea08a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87a971d9b65d4b298fa97db00ad8b393'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:35.792607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1258/2494] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:35 2022] DELETE /identity/v3/users/87a971d9b65d4b298fa97db00ad8b393 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:36.243215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:36.246113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:36.287213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2e7d6-bc54-46b0-95f8-e796b5ea08a0 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:36.290606 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8ef2e7d6-bc54-46b0-95f8-e796b5ea08a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:36.294878 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:36.414739 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2e7d6-bc54-46b0-95f8-e796b5ea08a0 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:36.420965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb2e59-3e8d-4f1c-b32a-3fd27b090db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:36.421837 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb2e59-3e8d-4f1c-b32a-3fd27b090db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:36.422359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb2e59-3e8d-4f1c-b32a-3fd27b090db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:36.422687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb2e59-3e8d-4f1c-b32a-3fd27b090db3 admin admin] PATH_INFO: `/v3/projects/1b7617dac3914c90b31955da1108babf` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:36.424279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb2e59-3e8d-4f1c-b32a-3fd27b090db3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b7617dac3914c90b31955da1108babf)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:36.427145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb2e59-3e8d-4f1c-b32a-3fd27b090db3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:36.448055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e8da2-dd0b-4cdf-b645-c9fba2c83407 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:36.496282 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0beb2e59-3e8d-4f1c-b32a-3fd27b090db3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b7617dac3914c90b31955da1108babf', 'request_id': 'req-0beb2e59-3e8d-4f1c-b32a-3fd27b090db3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:36.504931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0beb2e59-3e8d-4f1c-b32a-3fd27b090db3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b7617dac3914c90b31955da1108babf', 'request_id': 'req-0beb2e59-3e8d-4f1c-b32a-3fd27b090db3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:36.521276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0beb2e59-3e8d-4f1c-b32a-3fd27b090db3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b7617dac3914c90b31955da1108babf', 'request_id': 'req-0beb2e59-3e8d-4f1c-b32a-3fd27b090db3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:36.549941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0beb2e59-3e8d-4f1c-b32a-3fd27b090db3 admin admin] The token cache is being invalidate because project 1b7617dac3914c90b31955da1108babf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:36.549941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0beb2e59-3e8d-4f1c-b32a-3fd27b090db3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:36.551262 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0beb2e59-3e8d-4f1c-b32a-3fd27b090db3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:36.554132 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1259/2495] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:36 2022] DELETE /identity/v3/projects/1b7617dac3914c90b31955da1108babf => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:36.563562 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f456b99-630a-4214-9444-1c9be89e4b08 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:36.564846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f456b99-630a-4214-9444-1c9be89e4b08 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:36.565848 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f456b99-630a-4214-9444-1c9be89e4b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:36.605822 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c96663-5cdf-44ee-8d7d-d0b90a3bc58c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e642dd7a07ca4afaa40c2769299d8bad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:36.605822 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c96663-5cdf-44ee-8d7d-d0b90a3bc58c service placement] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:36.605822 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c96663-5cdf-44ee-8d7d-d0b90a3bc58c service placement] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:36.606448 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c96663-5cdf-44ee-8d7d-d0b90a3bc58c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:36.606752 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-1f456b99-630a-4214-9444-1c9be89e4b08 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:36.608933 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d0c96663-5cdf-44ee-8d7d-d0b90a3bc58c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:36.609766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1f456b99-630a-4214-9444-1c9be89e4b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:36.630353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:36.640946 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c96663-5cdf-44ee-8d7d-d0b90a3bc58c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:36.644986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c96663-5cdf-44ee-8d7d-d0b90a3bc58c service placement] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:36.684110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1237/2496] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:36.777558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1260/2497] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:36.789899 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b81b0f-d918-4ec9-881d-35c35c10d274 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:36.790306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b81b0f-d918-4ec9-881d-35c35c10d274 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:36.791349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b81b0f-d918-4ec9-881d-35c35c10d274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:36.825839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-62b81b0f-d918-4ec9-881d-35c35c10d274 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:36.827182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-62b81b0f-d918-4ec9-881d-35c35c10d274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:36.848454 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1238/2498] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:36.854922 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f456b99-630a-4214-9444-1c9be89e4b08 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:36.873616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f456b99-630a-4214-9444-1c9be89e4b08 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:36.883165 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b5901-263e-44d1-acc9-7b025256866b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:36.883970 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b5901-263e-44d1-acc9-7b025256866b admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:36.884258 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b5901-263e-44d1-acc9-7b025256866b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:36.884537 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b5901-263e-44d1-acc9-7b025256866b admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:36.885237 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b5901-263e-44d1-acc9-7b025256866b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:32:36.885580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b5901-263e-44d1-acc9-7b025256866b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:36.887228 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b5901-263e-44d1-acc9-7b025256866b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:36.901746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1261/2499] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:32:36.910677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b81b0f-d918-4ec9-881d-35c35c10d274 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:36.923568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b81b0f-d918-4ec9-881d-35c35c10d274 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:36.930127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb7d19-a434-4300-a767-1d28925a0d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:36.930938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb7d19-a434-4300-a767-1d28925a0d7f admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:36.931385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb7d19-a434-4300-a767-1d28925a0d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:36.931829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb7d19-a434-4300-a767-1d28925a0d7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:36.932610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb7d19-a434-4300-a767-1d28925a0d7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:36.934922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb7d19-a434-4300-a767-1d28925a0d7f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:36.982725 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1239/2500] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:36 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:36.988756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b5901-263e-44d1-acc9-7b025256866b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:37.009269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b5901-263e-44d1-acc9-7b025256866b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:37.113938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e4ff9-f999-4e20-912e-787eeedcc337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:37.115019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e4ff9-f999-4e20-912e-787eeedcc337 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:37.115500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e4ff9-f999-4e20-912e-787eeedcc337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:37.115987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e4ff9-f999-4e20-912e-787eeedcc337 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:37.117199 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e4ff9-f999-4e20-912e-787eeedcc337 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:37.120556 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e4ff9-f999-4e20-912e-787eeedcc337 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:37.165803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1262/2501] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:36 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:37.172730 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb7d19-a434-4300-a767-1d28925a0d7f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:37.191123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb7d19-a434-4300-a767-1d28925a0d7f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:37.196585 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ad0a2-6879-4b3e-b700-6aebf8809795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:37.197256 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ad0a2-6879-4b3e-b700-6aebf8809795 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:37.197582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ad0a2-6879-4b3e-b700-6aebf8809795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:37.197810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ad0a2-6879-4b3e-b700-6aebf8809795 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:37.198506 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ad0a2-6879-4b3e-b700-6aebf8809795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:37.200387 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ad0a2-6879-4b3e-b700-6aebf8809795 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:37.223353 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1240/2502] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:37.230602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e4ff9-f999-4e20-912e-787eeedcc337 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:37.243634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e4ff9-f999-4e20-912e-787eeedcc337 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:37.252440 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e143ee-a031-471e-80d2-266ba55f8ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:37.253225 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e143ee-a031-471e-80d2-266ba55f8ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:37.253519 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e143ee-a031-471e-80d2-266ba55f8ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:37.253858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e143ee-a031-471e-80d2-266ba55f8ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:37.254542 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e143ee-a031-471e-80d2-266ba55f8ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:37.256333 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e143ee-a031-471e-80d2-266ba55f8ad0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:37.275971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1263/2503] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:37.277912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ad0a2-6879-4b3e-b700-6aebf8809795 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:37.291652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ad0a2-6879-4b3e-b700-6aebf8809795 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:37.300743 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf4616-8143-497e-9b61-bdf0529a05eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:37.302201 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf4616-8143-497e-9b61-bdf0529a05eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:37.302663 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf4616-8143-497e-9b61-bdf0529a05eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:37.303300 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf4616-8143-497e-9b61-bdf0529a05eb admin admin] PATH_INFO: `/v3/projects/87877dbc4a8a454182f279c412275476/users/cde260a102a14fd6848821e3cb3cba3e/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:37.333458 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf4616-8143-497e-9b61-bdf0529a05eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=87877dbc4a8a454182f279c412275476, user_id=cde260a102a14fd6848821e3cb3cba3e, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:37.336488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf4616-8143-497e-9b61-bdf0529a05eb admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:37.366489 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1241/2504] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:37 2022] PUT /identity/v3/projects/87877dbc4a8a454182f279c412275476/users/cde260a102a14fd6848821e3cb3cba3e/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:37.371061 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c7209-9748-48d0-b8ae-8b815f6d15ac None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:37.371500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c7209-9748-48d0-b8ae-8b815f6d15ac None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:37.371929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c7209-9748-48d0-b8ae-8b815f6d15ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:37.428500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c81c7209-9748-48d0-b8ae-8b815f6d15ac None None] MFA Rules not processed for user `cde260a102a14fd6848821e3cb3cba3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:37.433807 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c81c7209-9748-48d0-b8ae-8b815f6d15ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:37.527321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1264/2505] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:37.545342 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf4616-8143-497e-9b61-bdf0529a05eb admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:37.548175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4baf4616-8143-497e-9b61-bdf0529a05eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:37.601045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:37.654512 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf4616-8143-497e-9b61-bdf0529a05eb admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:37.660245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8025db-b0e9-441e-8e31-c38f089a8b9b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2211e237fe554a82baf129e440a0fb43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:37.660888 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8025db-b0e9-441e-8e31-c38f089a8b9b service glance] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:37.661217 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8025db-b0e9-441e-8e31-c38f089a8b9b service glance] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:37.661501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8025db-b0e9-441e-8e31-c38f089a8b9b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:37.675814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8025db-b0e9-441e-8e31-c38f089a8b9b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:37.678679 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8025db-b0e9-441e-8e31-c38f089a8b9b service glance] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:37.697825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1242/2506] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:42.973390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c7209-9748-48d0-b8ae-8b815f6d15ac None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:42.975796 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c81c7209-9748-48d0-b8ae-8b815f6d15ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:42.994933 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:43.060261 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c7209-9748-48d0-b8ae-8b815f6d15ac None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:43.065646 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca8e66-8a2e-4951-8071-81cf2d64de09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:43.066319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca8e66-8a2e-4951-8071-81cf2d64de09 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:43.066595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca8e66-8a2e-4951-8071-81cf2d64de09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:43.066871 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca8e66-8a2e-4951-8071-81cf2d64de09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:43.069015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c5ca8e66-8a2e-4951-8071-81cf2d64de09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:43.151245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca8e66-8a2e-4951-8071-81cf2d64de09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:43.152763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca8e66-8a2e-4951-8071-81cf2d64de09 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:43.163785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1265/2507] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:47.103764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8025db-b0e9-441e-8e31-c38f089a8b9b service glance] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:47.105263 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0a8025db-b0e9-441e-8e31-c38f089a8b9b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:47.121631 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8025db-b0e9-441e-8e31-c38f089a8b9b service glance] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:47.127019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538ba9f-cc64-4f03-b260-865ec87385e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:47.127658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538ba9f-cc64-4f03-b260-865ec87385e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:47.127937 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538ba9f-cc64-4f03-b260-865ec87385e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:47.128212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538ba9f-cc64-4f03-b260-865ec87385e9 admin admin] PATH_INFO: `/v3/users/1db1a9a77be6452991ec4f99af814e7a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:47.129405 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538ba9f-cc64-4f03-b260-865ec87385e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1db1a9a77be6452991ec4f99af814e7a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:47.131337 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538ba9f-cc64-4f03-b260-865ec87385e9 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:47.182714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4538ba9f-cc64-4f03-b260-865ec87385e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1db1a9a77be6452991ec4f99af814e7a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:47.183416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4538ba9f-cc64-4f03-b260-865ec87385e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1db1a9a77be6452991ec4f99af814e7a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:47.193326 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4538ba9f-cc64-4f03-b260-865ec87385e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1db1a9a77be6452991ec4f99af814e7a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:47.208498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4538ba9f-cc64-4f03-b260-865ec87385e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1db1a9a77be6452991ec4f99af814e7a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:47.217916 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4538ba9f-cc64-4f03-b260-865ec87385e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1db1a9a77be6452991ec4f99af814e7a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:47.229788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1243/2508] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:47 2022] DELETE /identity/v3/users/1db1a9a77be6452991ec4f99af814e7a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:47.439987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca8e66-8a2e-4951-8071-81cf2d64de09 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:47.450956 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca8e66-8a2e-4951-8071-81cf2d64de09 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:47.528151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ce7a7-fe2a-4ce5-a1d1-0cd5198ca0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:47.528837 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ce7a7-fe2a-4ce5-a1d1-0cd5198ca0c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:47.529153 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ce7a7-fe2a-4ce5-a1d1-0cd5198ca0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:47.529434 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ce7a7-fe2a-4ce5-a1d1-0cd5198ca0c8 admin admin] PATH_INFO: `/v3/projects/c0fc1c15f821495e9883814ffc7aed78` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:47.531263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ce7a7-fe2a-4ce5-a1d1-0cd5198ca0c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0fc1c15f821495e9883814ffc7aed78)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:47.533666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ce7a7-fe2a-4ce5-a1d1-0cd5198ca0c8 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:47.587496 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a34ce7a7-fe2a-4ce5-a1d1-0cd5198ca0c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0fc1c15f821495e9883814ffc7aed78', 'request_id': 'req-a34ce7a7-fe2a-4ce5-a1d1-0cd5198ca0c8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:47.596529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a34ce7a7-fe2a-4ce5-a1d1-0cd5198ca0c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0fc1c15f821495e9883814ffc7aed78', 'request_id': 'req-a34ce7a7-fe2a-4ce5-a1d1-0cd5198ca0c8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:47.601034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:47.606667 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a34ce7a7-fe2a-4ce5-a1d1-0cd5198ca0c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0fc1c15f821495e9883814ffc7aed78', 'request_id': 'req-a34ce7a7-fe2a-4ce5-a1d1-0cd5198ca0c8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:47.623186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a34ce7a7-fe2a-4ce5-a1d1-0cd5198ca0c8 admin admin] The token cache is being invalidate because project c0fc1c15f821495e9883814ffc7aed78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:47.623585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a34ce7a7-fe2a-4ce5-a1d1-0cd5198ca0c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:47.624297 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a34ce7a7-fe2a-4ce5-a1d1-0cd5198ca0c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:47.626234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1266/2509] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:47 2022] DELETE /identity/v3/projects/c0fc1c15f821495e9883814ffc7aed78 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:47.631925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d4898-ee0e-4cee-9f80-4cfd0f997481 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:47.632236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d4898-ee0e-4cee-9f80-4cfd0f997481 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:47.632580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d4898-ee0e-4cee-9f80-4cfd0f997481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:47.653096 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-6a9d4898-ee0e-4cee-9f80-4cfd0f997481 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:47.654317 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6a9d4898-ee0e-4cee-9f80-4cfd0f997481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:47.728716 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1244/2510] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:47.737679 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d41c74-7d40-4c8f-93dd-9b9fd8863460 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:47.737969 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d41c74-7d40-4c8f-93dd-9b9fd8863460 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:47.738249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d41c74-7d40-4c8f-93dd-9b9fd8863460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:47.760830 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-08d41c74-7d40-4c8f-93dd-9b9fd8863460 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:47.762112 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-08d41c74-7d40-4c8f-93dd-9b9fd8863460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:47.771685 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1267/2511] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:47.776043 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d4898-ee0e-4cee-9f80-4cfd0f997481 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:47.789210 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d4898-ee0e-4cee-9f80-4cfd0f997481 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:47.792612 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86afa7-bf9e-4493-9eb3-c460a55e4492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:47.793307 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86afa7-bf9e-4493-9eb3-c460a55e4492 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:47.793593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86afa7-bf9e-4493-9eb3-c460a55e4492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:47.794025 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86afa7-bf9e-4493-9eb3-c460a55e4492 admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:47.794806 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86afa7-bf9e-4493-9eb3-c460a55e4492 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:32:47.795007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86afa7-bf9e-4493-9eb3-c460a55e4492 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:47.796766 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86afa7-bf9e-4493-9eb3-c460a55e4492 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:47.818067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1245/2512] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:32:47.823744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d41c74-7d40-4c8f-93dd-9b9fd8863460 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:47.841377 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d41c74-7d40-4c8f-93dd-9b9fd8863460 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:47.849605 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cdba4-b5dd-443c-af06-9a48f2357c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:47.850529 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cdba4-b5dd-443c-af06-9a48f2357c50 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:47.851022 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cdba4-b5dd-443c-af06-9a48f2357c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:47.851411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cdba4-b5dd-443c-af06-9a48f2357c50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:47.852566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cdba4-b5dd-443c-af06-9a48f2357c50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:47.856484 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cdba4-b5dd-443c-af06-9a48f2357c50 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:47.895822 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1268/2513] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:47 2022] POST /identity/v3/projects => generated 405 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:47.900133 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86afa7-bf9e-4493-9eb3-c460a55e4492 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:47.910532 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86afa7-bf9e-4493-9eb3-c460a55e4492 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:47.943553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:47.976159 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc9901-4115-44ad-8671-6ca023d8008e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:47.976983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc9901-4115-44ad-8671-6ca023d8008e admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:47.977319 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc9901-4115-44ad-8671-6ca023d8008e admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:47.977597 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc9901-4115-44ad-8671-6ca023d8008e admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:47.978493 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc9901-4115-44ad-8671-6ca023d8008e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:47.980758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc9901-4115-44ad-8671-6ca023d8008e admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:48.006061 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1246/2514] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:47 2022] POST /identity/v3/users => generated 303 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:48.010741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cdba4-b5dd-443c-af06-9a48f2357c50 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:48.022213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cdba4-b5dd-443c-af06-9a48f2357c50 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:48.027211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2558963-8739-444e-be44-24b4c2162e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:48.027940 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2558963-8739-444e-be44-24b4c2162e2f admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:48.028111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2558963-8739-444e-be44-24b4c2162e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:48.028413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2558963-8739-444e-be44-24b4c2162e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:48.029094 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2558963-8739-444e-be44-24b4c2162e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:48.031005 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2558963-8739-444e-be44-24b4c2162e2f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:48.042014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1269/2515] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:48.048274 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc9901-4115-44ad-8671-6ca023d8008e admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:48.062894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc9901-4115-44ad-8671-6ca023d8008e admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:48.069333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973be95e-a4e5-4d7c-8b79-4a65d140ff73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:48.070022 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973be95e-a4e5-4d7c-8b79-4a65d140ff73 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:48.070481 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973be95e-a4e5-4d7c-8b79-4a65d140ff73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:48.070835 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973be95e-a4e5-4d7c-8b79-4a65d140ff73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:48.071696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973be95e-a4e5-4d7c-8b79-4a65d140ff73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:48.073741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973be95e-a4e5-4d7c-8b79-4a65d140ff73 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:48.087137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1247/2516] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:48.092441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2558963-8739-444e-be44-24b4c2162e2f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:48.108163 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2558963-8739-444e-be44-24b4c2162e2f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:48.114115 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96845bf2-9f44-4e8f-97e4-8dbe9a0a0164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:48.114784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96845bf2-9f44-4e8f-97e4-8dbe9a0a0164 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:48.115069 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96845bf2-9f44-4e8f-97e4-8dbe9a0a0164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:48.115347 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96845bf2-9f44-4e8f-97e4-8dbe9a0a0164 admin admin] PATH_INFO: `/v3/projects/c5168e9113384226a90b5bfd2fe46a9c/users/0d711e5d1c2b4d00b80b45fc1ce05863/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:48.132283 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96845bf2-9f44-4e8f-97e4-8dbe9a0a0164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5168e9113384226a90b5bfd2fe46a9c, user_id=0d711e5d1c2b4d00b80b45fc1ce05863, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:48.135007 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96845bf2-9f44-4e8f-97e4-8dbe9a0a0164 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:48.149426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1270/2517] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:48 2022] PUT /identity/v3/projects/c5168e9113384226a90b5bfd2fe46a9c/users/0d711e5d1c2b4d00b80b45fc1ce05863/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:48.154579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf8711-75e7-4421-b7aa-c39c5c472a14 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:48.155111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf8711-75e7-4421-b7aa-c39c5c472a14 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:48.155305 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf8711-75e7-4421-b7aa-c39c5c472a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:48.176961 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-debf8711-75e7-4421-b7aa-c39c5c472a14 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:48.178361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-debf8711-75e7-4421-b7aa-c39c5c472a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:48.262763 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1248/2518] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:48.273953 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96845bf2-9f44-4e8f-97e4-8dbe9a0a0164 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:48.275756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-96845bf2-9f44-4e8f-97e4-8dbe9a0a0164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:48.340450 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96845bf2-9f44-4e8f-97e4-8dbe9a0a0164 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:48.345680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b07517-aade-4279-af47-89f3497f7dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:48.346341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b07517-aade-4279-af47-89f3497f7dab service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:48.346661 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b07517-aade-4279-af47-89f3497f7dab service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:48.346942 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b07517-aade-4279-af47-89f3497f7dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:48.359449 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b07517-aade-4279-af47-89f3497f7dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:48.361049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b07517-aade-4279-af47-89f3497f7dab service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:48.373365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1271/2519] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:49.217301 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c25527-c806-41f7-b0e5-1b69892a4ae1 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:49.218538 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c25527-c806-41f7-b0e5-1b69892a4ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:49.218538 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c25527-c806-41f7-b0e5-1b69892a4ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:49.254827 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-a8c25527-c806-41f7-b0e5-1b69892a4ae1 None None] MFA Rules not processed for user `0d711e5d1c2b4d00b80b45fc1ce05863`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:49.255976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a8c25527-c806-41f7-b0e5-1b69892a4ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:49.346371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1249/2520] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:49 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:49.351634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b07517-aade-4279-af47-89f3497f7dab service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:49.364835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b07517-aade-4279-af47-89f3497f7dab service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:49.371782 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd881994-f2cd-40fe-a18e-d09d723b0a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:49.372419 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd881994-f2cd-40fe-a18e-d09d723b0a79 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:49.372857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd881994-f2cd-40fe-a18e-d09d723b0a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:49.373169 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd881994-f2cd-40fe-a18e-d09d723b0a79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:49.373828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd881994-f2cd-40fe-a18e-d09d723b0a79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:49.375921 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd881994-f2cd-40fe-a18e-d09d723b0a79 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:49.417352 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1272/2521] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:49 2022] POST /identity/v3/projects => generated 403 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:49.418885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c25527-c806-41f7-b0e5-1b69892a4ae1 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:49.432739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c25527-c806-41f7-b0e5-1b69892a4ae1 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:49.475291 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:49.500119 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed75959-0a85-4fc7-a01a-48f17be9f79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:49.500906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed75959-0a85-4fc7-a01a-48f17be9f79d admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:49.501233 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed75959-0a85-4fc7-a01a-48f17be9f79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:49.501512 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed75959-0a85-4fc7-a01a-48f17be9f79d admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:49.502176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed75959-0a85-4fc7-a01a-48f17be9f79d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:49.504196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed75959-0a85-4fc7-a01a-48f17be9f79d admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:49.528933 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1250/2522] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:49 2022] POST /identity/v3/users => generated 308 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:49.533931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd881994-f2cd-40fe-a18e-d09d723b0a79 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:49.545950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd881994-f2cd-40fe-a18e-d09d723b0a79 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:49.551415 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fe336-423c-4ef2-a1d4-2adb04a78e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:49.552132 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fe336-423c-4ef2-a1d4-2adb04a78e2e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:49.552414 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fe336-423c-4ef2-a1d4-2adb04a78e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:49.552695 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fe336-423c-4ef2-a1d4-2adb04a78e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:49.553374 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5fe336-423c-4ef2-a1d4-2adb04a78e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:49.555006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5fe336-423c-4ef2-a1d4-2adb04a78e2e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:49.567230 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1273/2523] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:49.572295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed75959-0a85-4fc7-a01a-48f17be9f79d admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:49.584037 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed75959-0a85-4fc7-a01a-48f17be9f79d admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:49.588800 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853c1b2-439d-4389-a209-99af759274cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:49.590927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853c1b2-439d-4389-a209-99af759274cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:49.590927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853c1b2-439d-4389-a209-99af759274cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:49.590927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853c1b2-439d-4389-a209-99af759274cb admin admin] PATH_INFO: `/v3/projects/3d5c045dafd64c6cb15078633a913fba/users/d9175749adfb47d18da37d4e0c15d1db/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:49.610193 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6853c1b2-439d-4389-a209-99af759274cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d5c045dafd64c6cb15078633a913fba, user_id=d9175749adfb47d18da37d4e0c15d1db, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:49.613892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6853c1b2-439d-4389-a209-99af759274cb admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:49.633858 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1251/2524] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:49 2022] PUT /identity/v3/projects/3d5c045dafd64c6cb15078633a913fba/users/d9175749adfb47d18da37d4e0c15d1db/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:49.639239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fe336-423c-4ef2-a1d4-2adb04a78e2e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:49.651411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fe336-423c-4ef2-a1d4-2adb04a78e2e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:49.709330 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:49.717845 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a885697-adb1-48e7-8df1-91a7aa8cb9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:49.719296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a885697-adb1-48e7-8df1-91a7aa8cb9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:49.719864 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a885697-adb1-48e7-8df1-91a7aa8cb9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:49.720287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a885697-adb1-48e7-8df1-91a7aa8cb9d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:49.721072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a885697-adb1-48e7-8df1-91a7aa8cb9d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:49.722811 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a885697-adb1-48e7-8df1-91a7aa8cb9d4 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:49.735321 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1274/2525] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:32:49.741009 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853c1b2-439d-4389-a209-99af759274cb admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:49.752811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853c1b2-439d-4389-a209-99af759274cb admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:49.757754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34e03a-dd12-4b6b-a80f-b8be62bfd1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:49.758460 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34e03a-dd12-4b6b-a80f-b8be62bfd1e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:49.758830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34e03a-dd12-4b6b-a80f-b8be62bfd1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:49.759111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34e03a-dd12-4b6b-a80f-b8be62bfd1e9 admin admin] PATH_INFO: `/v3/projects/3d5c045dafd64c6cb15078633a913fba/users/d9175749adfb47d18da37d4e0c15d1db/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:49.761178 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34e03a-dd12-4b6b-a80f-b8be62bfd1e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d5c045dafd64c6cb15078633a913fba, user_id=d9175749adfb47d18da37d4e0c15d1db, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:49.763871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34e03a-dd12-4b6b-a80f-b8be62bfd1e9 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:49.781871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1252/2526] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:49 2022] PUT /identity/v3/projects/3d5c045dafd64c6cb15078633a913fba/users/d9175749adfb47d18da37d4e0c15d1db/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:50.475207 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8812c-fbe0-4abd-8dc7-e751b25264c8 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:50.475748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8812c-fbe0-4abd-8dc7-e751b25264c8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:50.475748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8812c-fbe0-4abd-8dc7-e751b25264c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:50.510061 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-58e8812c-fbe0-4abd-8dc7-e751b25264c8 None None] MFA Rules not processed for user `d9175749adfb47d18da37d4e0c15d1db`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:50.511314 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-58e8812c-fbe0-4abd-8dc7-e751b25264c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:50.613455 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1275/2527] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:50 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:51.953491 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34e03a-dd12-4b6b-a80f-b8be62bfd1e9 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:52.080003 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34e03a-dd12-4b6b-a80f-b8be62bfd1e9 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:52.085946 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb694c6-7b68-413e-bcf9-62fcfcbd096d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:52.086714 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb694c6-7b68-413e-bcf9-62fcfcbd096d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:52.087066 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb694c6-7b68-413e-bcf9-62fcfcbd096d admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:52.087391 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb694c6-7b68-413e-bcf9-62fcfcbd096d admin admin] PATH_INFO: `/v3/users/0d711e5d1c2b4d00b80b45fc1ce05863` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:52.088902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb694c6-7b68-413e-bcf9-62fcfcbd096d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d711e5d1c2b4d00b80b45fc1ce05863)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:52.091198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb694c6-7b68-413e-bcf9-62fcfcbd096d admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:52.184702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-dcb694c6-7b68-413e-bcf9-62fcfcbd096d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d711e5d1c2b4d00b80b45fc1ce05863'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.185627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-dcb694c6-7b68-413e-bcf9-62fcfcbd096d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d711e5d1c2b4d00b80b45fc1ce05863'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.195882 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-dcb694c6-7b68-413e-bcf9-62fcfcbd096d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d711e5d1c2b4d00b80b45fc1ce05863'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.216102 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-dcb694c6-7b68-413e-bcf9-62fcfcbd096d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d711e5d1c2b4d00b80b45fc1ce05863'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.232226 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-dcb694c6-7b68-413e-bcf9-62fcfcbd096d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d711e5d1c2b4d00b80b45fc1ce05863'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.244719 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1253/2528] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:51 2022] DELETE /identity/v3/users/0d711e5d1c2b4d00b80b45fc1ce05863 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:52.583300 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8812c-fbe0-4abd-8dc7-e751b25264c8 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:52.604234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8812c-fbe0-4abd-8dc7-e751b25264c8 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:52.710997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:52.711987 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:52.712493 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:52.713010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d admin admin] PATH_INFO: `/v3/projects/c5168e9113384226a90b5bfd2fe46a9c` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:52.715106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5168e9113384226a90b5bfd2fe46a9c)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:52.718722 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:52.783677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5168e9113384226a90b5bfd2fe46a9c', 'request_id': 'req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.797143 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5168e9113384226a90b5bfd2fe46a9c', 'request_id': 'req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.820133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5168e9113384226a90b5bfd2fe46a9c', 'request_id': 'req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.836100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d admin admin] The token cache is being invalidate because project c5168e9113384226a90b5bfd2fe46a9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:52.836439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.838453 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.839018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1276/2529] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:52 2022] DELETE /identity/v3/projects/c5168e9113384226a90b5bfd2fe46a9c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:52.843290 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb694c6-7b68-413e-bcf9-62fcfcbd096d admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:52.844850 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-dcb694c6-7b68-413e-bcf9-62fcfcbd096d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:52.860380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb694c6-7b68-413e-bcf9-62fcfcbd096d admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:52.871434 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:52.962439 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e592d-edeb-4de7-9e90-0b406050841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:52.963319 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e592d-edeb-4de7-9e90-0b406050841d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:52.963478 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e592d-edeb-4de7-9e90-0b406050841d admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:52.963645 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e592d-edeb-4de7-9e90-0b406050841d admin admin] PATH_INFO: `/v3/users/d9175749adfb47d18da37d4e0c15d1db` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:52.965277 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e592d-edeb-4de7-9e90-0b406050841d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9175749adfb47d18da37d4e0c15d1db)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:52.968510 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e592d-edeb-4de7-9e90-0b406050841d admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:53.036150 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f29e592d-edeb-4de7-9e90-0b406050841d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9175749adfb47d18da37d4e0c15d1db'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:53.037121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f29e592d-edeb-4de7-9e90-0b406050841d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9175749adfb47d18da37d4e0c15d1db'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:53.047215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f29e592d-edeb-4de7-9e90-0b406050841d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9175749adfb47d18da37d4e0c15d1db'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:53.064493 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f29e592d-edeb-4de7-9e90-0b406050841d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9175749adfb47d18da37d4e0c15d1db'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:53.079028 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f29e592d-edeb-4de7-9e90-0b406050841d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9175749adfb47d18da37d4e0c15d1db'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:53.091314 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1254/2530] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:52 2022] DELETE /identity/v3/users/d9175749adfb47d18da37d4e0c15d1db => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:53.320881 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:53.322509 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:53.341233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbb685-4b4f-4b0d-817c-76a6c9c3dd4d admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:53.353772 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:53.415869 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd2100-02ef-4adf-8784-b53bba2aeeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:53.416513 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd2100-02ef-4adf-8784-b53bba2aeeaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:53.416794 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd2100-02ef-4adf-8784-b53bba2aeeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:53.417088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd2100-02ef-4adf-8784-b53bba2aeeaf admin admin] PATH_INFO: `/v3/projects/3d5c045dafd64c6cb15078633a913fba` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:53.418234 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd2100-02ef-4adf-8784-b53bba2aeeaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d5c045dafd64c6cb15078633a913fba)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:53.420149 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd2100-02ef-4adf-8784-b53bba2aeeaf admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:53.479694 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-adfd2100-02ef-4adf-8784-b53bba2aeeaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d5c045dafd64c6cb15078633a913fba', 'request_id': 'req-adfd2100-02ef-4adf-8784-b53bba2aeeaf'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:53.486756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-adfd2100-02ef-4adf-8784-b53bba2aeeaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d5c045dafd64c6cb15078633a913fba', 'request_id': 'req-adfd2100-02ef-4adf-8784-b53bba2aeeaf'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:53.496018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-adfd2100-02ef-4adf-8784-b53bba2aeeaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d5c045dafd64c6cb15078633a913fba', 'request_id': 'req-adfd2100-02ef-4adf-8784-b53bba2aeeaf'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:53.509115 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-adfd2100-02ef-4adf-8784-b53bba2aeeaf admin admin] The token cache is being invalidate because project 3d5c045dafd64c6cb15078633a913fba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:53.509417 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-adfd2100-02ef-4adf-8784-b53bba2aeeaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:53.510070 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-adfd2100-02ef-4adf-8784-b53bba2aeeaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:53.511569 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1277/2531] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:53 2022] DELETE /identity/v3/projects/3d5c045dafd64c6cb15078633a913fba => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:53.516918 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc24f7d-0fee-44ee-9fc3-5ee44709cab2 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:53.517248 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc24f7d-0fee-44ee-9fc3-5ee44709cab2 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:53.517561 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc24f7d-0fee-44ee-9fc3-5ee44709cab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:53.549095 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5cc24f7d-0fee-44ee-9fc3-5ee44709cab2 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:53.551667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5cc24f7d-0fee-44ee-9fc3-5ee44709cab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:53.680849 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1255/2532] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:53.694648 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494c9fc-97be-4765-b7fb-7005a1fce2b6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:53.695543 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494c9fc-97be-4765-b7fb-7005a1fce2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:53.695543 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494c9fc-97be-4765-b7fb-7005a1fce2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:53.725235 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-5494c9fc-97be-4765-b7fb-7005a1fce2b6 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:53.727502 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5494c9fc-97be-4765-b7fb-7005a1fce2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:53.742213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1278/2533] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:53.748539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc24f7d-0fee-44ee-9fc3-5ee44709cab2 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:53.761902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc24f7d-0fee-44ee-9fc3-5ee44709cab2 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:53.771553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc326a-e55b-4bda-b906-c3fd8452e0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:53.772703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc326a-e55b-4bda-b906-c3fd8452e0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:53.773271 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc326a-e55b-4bda-b906-c3fd8452e0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:53.773792 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc326a-e55b-4bda-b906-c3fd8452e0e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:53.775124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc326a-e55b-4bda-b906-c3fd8452e0e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:32:53.775905 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc326a-e55b-4bda-b906-c3fd8452e0e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:53.779117 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc326a-e55b-4bda-b906-c3fd8452e0e1 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:53.805346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1256/2534] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:32:53.812498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494c9fc-97be-4765-b7fb-7005a1fce2b6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:53.823524 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494c9fc-97be-4765-b7fb-7005a1fce2b6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:53.830330 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79bd76-f3af-428a-bec2-e63a78519033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:53.831012 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79bd76-f3af-428a-bec2-e63a78519033 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:53.831332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79bd76-f3af-428a-bec2-e63a78519033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:53.831649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79bd76-f3af-428a-bec2-e63a78519033 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:53.832343 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79bd76-f3af-428a-bec2-e63a78519033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:53.834446 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79bd76-f3af-428a-bec2-e63a78519033 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:53.868737 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1279/2535] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:53 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:53.874959 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc326a-e55b-4bda-b906-c3fd8452e0e1 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:53.891233 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc326a-e55b-4bda-b906-c3fd8452e0e1 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:53.911415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:53.976873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7914695-d022-44e1-b62b-91bb371b6195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:53.977651 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7914695-d022-44e1-b62b-91bb371b6195 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:53.977768 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7914695-d022-44e1-b62b-91bb371b6195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:53.978068 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7914695-d022-44e1-b62b-91bb371b6195 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:53.979050 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7914695-d022-44e1-b62b-91bb371b6195 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:53.981360 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7914695-d022-44e1-b62b-91bb371b6195 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.007332 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1257/2536] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:53 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:54.012770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79bd76-f3af-428a-bec2-e63a78519033 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.032186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79bd76-f3af-428a-bec2-e63a78519033 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:54.040935 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a9507-9945-475a-b00d-d4712567eabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:54.042163 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a9507-9945-475a-b00d-d4712567eabe admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.042743 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a9507-9945-475a-b00d-d4712567eabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.043090 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a9507-9945-475a-b00d-d4712567eabe admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.043943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a9507-9945-475a-b00d-d4712567eabe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.046979 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a9507-9945-475a-b00d-d4712567eabe admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.059991 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1280/2537] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:54.066322 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7914695-d022-44e1-b62b-91bb371b6195 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.081380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7914695-d022-44e1-b62b-91bb371b6195 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:54.089104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84017899-5292-4c27-9bad-9e58d5b09321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:54.089751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84017899-5292-4c27-9bad-9e58d5b09321 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.090026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84017899-5292-4c27-9bad-9e58d5b09321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.090297 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84017899-5292-4c27-9bad-9e58d5b09321 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.090955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84017899-5292-4c27-9bad-9e58d5b09321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.092575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84017899-5292-4c27-9bad-9e58d5b09321 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.106833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1258/2538] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:32:54.112389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a9507-9945-475a-b00d-d4712567eabe admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.126931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a9507-9945-475a-b00d-d4712567eabe admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:54.134353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c1cbe-3a6c-41c1-a402-e5701dd245a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:54.135476 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c1cbe-3a6c-41c1-a402-e5701dd245a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.135944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c1cbe-3a6c-41c1-a402-e5701dd245a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.136447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c1cbe-3a6c-41c1-a402-e5701dd245a9 admin admin] PATH_INFO: `/v3/projects/357560bec2994339a00ac5b5935115a6/users/099560ec6e7048e18060a5c56d662e5e/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.159650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c1cbe-3a6c-41c1-a402-e5701dd245a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=357560bec2994339a00ac5b5935115a6, user_id=099560ec6e7048e18060a5c56d662e5e, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.163152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c1cbe-3a6c-41c1-a402-e5701dd245a9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.190378 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1281/2539] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:54 2022] PUT /identity/v3/projects/357560bec2994339a00ac5b5935115a6/users/099560ec6e7048e18060a5c56d662e5e/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:54.210476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbcb28-6e2a-4420-8f20-7fb6a687f33a None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.211000 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbcb28-6e2a-4420-8f20-7fb6a687f33a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.211513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbcb28-6e2a-4420-8f20-7fb6a687f33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.256434 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-0edbcb28-6e2a-4420-8f20-7fb6a687f33a None None] MFA Rules not processed for user `099560ec6e7048e18060a5c56d662e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:54.257752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0edbcb28-6e2a-4420-8f20-7fb6a687f33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:54.347124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1259/2540] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:54.352423 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c1cbe-3a6c-41c1-a402-e5701dd245a9 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.401343 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:54.429194 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c1cbe-3a6c-41c1-a402-e5701dd245a9 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:54.435748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f15e8-c0bc-4fca-8c8f-70b35db7eb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:54.437357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f15e8-c0bc-4fca-8c8f-70b35db7eb45 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.437751 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f15e8-c0bc-4fca-8c8f-70b35db7eb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.438135 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f15e8-c0bc-4fca-8c8f-70b35db7eb45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.438919 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f15e8-c0bc-4fca-8c8f-70b35db7eb45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.441078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f15e8-c0bc-4fca-8c8f-70b35db7eb45 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.471723 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1282/2541] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:54 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:54.476718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbcb28-6e2a-4420-8f20-7fb6a687f33a None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.488662 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbcb28-6e2a-4420-8f20-7fb6a687f33a None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:54.556080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a64053-0972-4570-bc98-67a42fabcb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:54.556693 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a64053-0972-4570-bc98-67a42fabcb0f admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.557083 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a64053-0972-4570-bc98-67a42fabcb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.557363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a64053-0972-4570-bc98-67a42fabcb0f admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.558032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a64053-0972-4570-bc98-67a42fabcb0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.559973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a64053-0972-4570-bc98-67a42fabcb0f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.596203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1260/2542] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:54 2022] POST /identity/v3/users => generated 300 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:54.602127 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f15e8-c0bc-4fca-8c8f-70b35db7eb45 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.620694 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f15e8-c0bc-4fca-8c8f-70b35db7eb45 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:54.629682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f127a63-bfe6-4a74-bb69-1a312e8434a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:54.630400 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f127a63-bfe6-4a74-bb69-1a312e8434a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.630884 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f127a63-bfe6-4a74-bb69-1a312e8434a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.631367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f127a63-bfe6-4a74-bb69-1a312e8434a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.632407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f127a63-bfe6-4a74-bb69-1a312e8434a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.634773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f127a63-bfe6-4a74-bb69-1a312e8434a7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.656015 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1283/2543] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:54.659539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a64053-0972-4570-bc98-67a42fabcb0f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.671667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a64053-0972-4570-bc98-67a42fabcb0f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:54.676903 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e69c62-2ebf-403a-92a7-0f1c2bf67818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:54.677799 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e69c62-2ebf-403a-92a7-0f1c2bf67818 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.678078 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e69c62-2ebf-403a-92a7-0f1c2bf67818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.678351 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e69c62-2ebf-403a-92a7-0f1c2bf67818 admin admin] PATH_INFO: `/v3/projects/3243325520f34e83afdc5e605fcab9f4/users/2e30fa81f1d14871a8a8dce0debe7700/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.692983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:54.697001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e69c62-2ebf-403a-92a7-0f1c2bf67818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3243325520f34e83afdc5e605fcab9f4, user_id=2e30fa81f1d14871a8a8dce0debe7700, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.701299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e69c62-2ebf-403a-92a7-0f1c2bf67818 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.721530 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1261/2544] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:54 2022] PUT /identity/v3/projects/3243325520f34e83afdc5e605fcab9f4/users/2e30fa81f1d14871a8a8dce0debe7700/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:54.729637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f127a63-bfe6-4a74-bb69-1a312e8434a7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.739087 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f127a63-bfe6-4a74-bb69-1a312e8434a7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:54.841531 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc06d45-f5ea-4961-8dfd-f08d579ccf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:54.842232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc06d45-f5ea-4961-8dfd-f08d579ccf94 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.842567 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc06d45-f5ea-4961-8dfd-f08d579ccf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.843467 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc06d45-f5ea-4961-8dfd-f08d579ccf94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.844501 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc06d45-f5ea-4961-8dfd-f08d579ccf94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.846282 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc06d45-f5ea-4961-8dfd-f08d579ccf94 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.860647 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1284/2545] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:32:54.865995 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e69c62-2ebf-403a-92a7-0f1c2bf67818 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.879284 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e69c62-2ebf-403a-92a7-0f1c2bf67818 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:54.888003 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad7cda-4e51-4a2a-98be-036207854d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:54.888796 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad7cda-4e51-4a2a-98be-036207854d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.889110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad7cda-4e51-4a2a-98be-036207854d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.889456 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad7cda-4e51-4a2a-98be-036207854d03 admin admin] PATH_INFO: `/v3/projects/3243325520f34e83afdc5e605fcab9f4/users/2e30fa81f1d14871a8a8dce0debe7700/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.893665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad7cda-4e51-4a2a-98be-036207854d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3243325520f34e83afdc5e605fcab9f4, user_id=2e30fa81f1d14871a8a8dce0debe7700, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.897750 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad7cda-4e51-4a2a-98be-036207854d03 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.912426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1262/2546] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:54 2022] PUT /identity/v3/projects/3243325520f34e83afdc5e605fcab9f4/users/2e30fa81f1d14871a8a8dce0debe7700/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:54.934231 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd89817-3f2b-44f3-aa86-8e6caa93e5ad None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.934881 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd89817-3f2b-44f3-aa86-8e6caa93e5ad None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.935499 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd89817-3f2b-44f3-aa86-8e6caa93e5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.994556 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-4dd89817-3f2b-44f3-aa86-8e6caa93e5ad None None] MFA Rules not processed for user `2e30fa81f1d14871a8a8dce0debe7700`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:54.996986 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4dd89817-3f2b-44f3-aa86-8e6caa93e5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:55.121908 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1285/2547] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:55.141341 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad7cda-4e51-4a2a-98be-036207854d03 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:55.144356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-94ad7cda-4e51-4a2a-98be-036207854d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:55.285755 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad7cda-4e51-4a2a-98be-036207854d03 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:55.290500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403c488-550b-44a4-b032-4f2b50afa71e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:55.291470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403c488-550b-44a4-b032-4f2b50afa71e service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.291761 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403c488-550b-44a4-b032-4f2b50afa71e service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.292039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403c488-550b-44a4-b032-4f2b50afa71e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.307440 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403c488-550b-44a4-b032-4f2b50afa71e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:55.310263 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403c488-550b-44a4-b032-4f2b50afa71e service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:55.324772 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1263/2548] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:55.364611 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd89817-3f2b-44f3-aa86-8e6caa93e5ad None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:55.411921 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:55.448521 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd89817-3f2b-44f3-aa86-8e6caa93e5ad None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:55.453952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e0b88-b9d0-4d0e-bc58-f3f6fbe73a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:55.454593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e0b88-b9d0-4d0e-bc58-f3f6fbe73a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.454873 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e0b88-b9d0-4d0e-bc58-f3f6fbe73a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.455144 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e0b88-b9d0-4d0e-bc58-f3f6fbe73a72 admin admin] PATH_INFO: `/v3/users/099560ec6e7048e18060a5c56d662e5e` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.456272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e0b88-b9d0-4d0e-bc58-f3f6fbe73a72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=099560ec6e7048e18060a5c56d662e5e)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:55.458411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e0b88-b9d0-4d0e-bc58-f3f6fbe73a72 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:55.527139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1a2e0b88-b9d0-4d0e-bc58-f3f6fbe73a72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '099560ec6e7048e18060a5c56d662e5e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:55.540555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1a2e0b88-b9d0-4d0e-bc58-f3f6fbe73a72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '099560ec6e7048e18060a5c56d662e5e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:55.551291 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1a2e0b88-b9d0-4d0e-bc58-f3f6fbe73a72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '099560ec6e7048e18060a5c56d662e5e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:55.551912 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1a2e0b88-b9d0-4d0e-bc58-f3f6fbe73a72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '099560ec6e7048e18060a5c56d662e5e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:55.559608 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1a2e0b88-b9d0-4d0e-bc58-f3f6fbe73a72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '099560ec6e7048e18060a5c56d662e5e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:55.566894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1286/2549] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:55 2022] DELETE /identity/v3/users/099560ec6e7048e18060a5c56d662e5e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:55.571775 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162054a6-55e5-4291-8b47-c2df3719261b None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.572365 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162054a6-55e5-4291-8b47-c2df3719261b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.572365 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162054a6-55e5-4291-8b47-c2df3719261b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.593427 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-162054a6-55e5-4291-8b47-c2df3719261b None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:55.594985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-162054a6-55e5-4291-8b47-c2df3719261b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:55.672610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1264/2550] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:55.684474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e0b88-b9d0-4d0e-bc58-f3f6fbe73a72 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:55.685924 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1a2e0b88-b9d0-4d0e-bc58-f3f6fbe73a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:55.772289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e0b88-b9d0-4d0e-bc58-f3f6fbe73a72 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:55.776786 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcf49b-82c2-4b3a-8e04-494bd38f070b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:55.777452 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcf49b-82c2-4b3a-8e04-494bd38f070b service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.777742 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcf49b-82c2-4b3a-8e04-494bd38f070b service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.778008 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcf49b-82c2-4b3a-8e04-494bd38f070b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.790101 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfcf49b-82c2-4b3a-8e04-494bd38f070b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:55.791578 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfcf49b-82c2-4b3a-8e04-494bd38f070b service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:55.803975 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1287/2551] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:56.064191 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162054a6-55e5-4291-8b47-c2df3719261b None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:56.065661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-162054a6-55e5-4291-8b47-c2df3719261b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:56.077841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162054a6-55e5-4291-8b47-c2df3719261b None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:56.082702 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23560c-9f36-4e37-8335-c2c704fea52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:56.083505 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23560c-9f36-4e37-8335-c2c704fea52f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:56.083781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23560c-9f36-4e37-8335-c2c704fea52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:56.084051 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23560c-9f36-4e37-8335-c2c704fea52f admin admin] PATH_INFO: `/v3/users/cde260a102a14fd6848821e3cb3cba3e` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:56.085215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23560c-9f36-4e37-8335-c2c704fea52f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cde260a102a14fd6848821e3cb3cba3e)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:56.087144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23560c-9f36-4e37-8335-c2c704fea52f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:56.141099 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcf49b-82c2-4b3a-8e04-494bd38f070b service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:56.143050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1bfcf49b-82c2-4b3a-8e04-494bd38f070b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:56.157502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f23560c-9f36-4e37-8335-c2c704fea52f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cde260a102a14fd6848821e3cb3cba3e'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.158557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f23560c-9f36-4e37-8335-c2c704fea52f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cde260a102a14fd6848821e3cb3cba3e'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.169423 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcf49b-82c2-4b3a-8e04-494bd38f070b service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:56.181539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f23560c-9f36-4e37-8335-c2c704fea52f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cde260a102a14fd6848821e3cb3cba3e'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.182938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265b280-8715-47ed-9020-9e5d75dd28ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:56.184117 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:56.185361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265b280-8715-47ed-9020-9e5d75dd28ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:56.187716 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265b280-8715-47ed-9020-9e5d75dd28ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:56.191116 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265b280-8715-47ed-9020-9e5d75dd28ec admin admin] PATH_INFO: `/v3/projects/357560bec2994339a00ac5b5935115a6` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:56.193762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265b280-8715-47ed-9020-9e5d75dd28ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=357560bec2994339a00ac5b5935115a6)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:56.196563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265b280-8715-47ed-9020-9e5d75dd28ec admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:56.214093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f23560c-9f36-4e37-8335-c2c704fea52f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cde260a102a14fd6848821e3cb3cba3e'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.227920 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4f23560c-9f36-4e37-8335-c2c704fea52f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cde260a102a14fd6848821e3cb3cba3e'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.241030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1265/2552] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:56 2022] DELETE /identity/v3/users/cde260a102a14fd6848821e3cb3cba3e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:56.251293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09291905-a5be-45df-88ff-6c03d37f13dc None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:56.251764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09291905-a5be-45df-88ff-6c03d37f13dc None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:56.252077 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09291905-a5be-45df-88ff-6c03d37f13dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:56.275142 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:56.277280 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7265b280-8715-47ed-9020-9e5d75dd28ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '357560bec2994339a00ac5b5935115a6', 'request_id': 'req-7265b280-8715-47ed-9020-9e5d75dd28ec'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.286178 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-09291905-a5be-45df-88ff-6c03d37f13dc None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:56.287566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-09291905-a5be-45df-88ff-6c03d37f13dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:56.287722 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7265b280-8715-47ed-9020-9e5d75dd28ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '357560bec2994339a00ac5b5935115a6', 'request_id': 'req-7265b280-8715-47ed-9020-9e5d75dd28ec'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.303990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7265b280-8715-47ed-9020-9e5d75dd28ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '357560bec2994339a00ac5b5935115a6', 'request_id': 'req-7265b280-8715-47ed-9020-9e5d75dd28ec'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.344909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7265b280-8715-47ed-9020-9e5d75dd28ec admin admin] The token cache is being invalidate because project 357560bec2994339a00ac5b5935115a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:56.345596 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7265b280-8715-47ed-9020-9e5d75dd28ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.346759 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7265b280-8715-47ed-9020-9e5d75dd28ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.349313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1288/2553] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:56 2022] DELETE /identity/v3/projects/357560bec2994339a00ac5b5935115a6 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:56.356954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265b280-8715-47ed-9020-9e5d75dd28ec admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:56.359840 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7265b280-8715-47ed-9020-9e5d75dd28ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:56.406249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1266/2554] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:56.418573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09291905-a5be-45df-88ff-6c03d37f13dc None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:56.421377 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-09291905-a5be-45df-88ff-6c03d37f13dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:56.471181 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265b280-8715-47ed-9020-9e5d75dd28ec admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:56.478607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdaa1eb-5f93-41f3-b79e-d1cd3d6eef5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:56.479696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaa1eb-5f93-41f3-b79e-d1cd3d6eef5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:56.480070 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaa1eb-5f93-41f3-b79e-d1cd3d6eef5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:56.480354 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaa1eb-5f93-41f3-b79e-d1cd3d6eef5d admin admin] PATH_INFO: `/v3/users/2e30fa81f1d14871a8a8dce0debe7700` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:56.482077 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdaa1eb-5f93-41f3-b79e-d1cd3d6eef5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e30fa81f1d14871a8a8dce0debe7700)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:56.485408 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdaa1eb-5f93-41f3-b79e-d1cd3d6eef5d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:56.519253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09291905-a5be-45df-88ff-6c03d37f13dc None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:56.523992 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d11f0-627b-4641-948d-c09469adf69e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:56.524662 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d11f0-627b-4641-948d-c09469adf69e service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:56.524946 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d11f0-627b-4641-948d-c09469adf69e service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:56.525266 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d11f0-627b-4641-948d-c09469adf69e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:56.537221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d11f0-627b-4641-948d-c09469adf69e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:56.538708 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d11f0-627b-4641-948d-c09469adf69e service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:56.549859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1267/2555] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:32:56.558442 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bcdaa1eb-5f93-41f3-b79e-d1cd3d6eef5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e30fa81f1d14871a8a8dce0debe7700'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.578737 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bcdaa1eb-5f93-41f3-b79e-d1cd3d6eef5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e30fa81f1d14871a8a8dce0debe7700'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.595694 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bcdaa1eb-5f93-41f3-b79e-d1cd3d6eef5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e30fa81f1d14871a8a8dce0debe7700'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.597137 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bcdaa1eb-5f93-41f3-b79e-d1cd3d6eef5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e30fa81f1d14871a8a8dce0debe7700'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.609269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bcdaa1eb-5f93-41f3-b79e-d1cd3d6eef5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e30fa81f1d14871a8a8dce0debe7700'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.618601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1289/2556] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:56 2022] DELETE /identity/v3/users/2e30fa81f1d14871a8a8dce0debe7700 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:57.064596 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d11f0-627b-4641-948d-c09469adf69e service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.066689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ea2d11f0-627b-4641-948d-c09469adf69e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:57.090076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d11f0-627b-4641-948d-c09469adf69e service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:57.119541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdaa1eb-5f93-41f3-b79e-d1cd3d6eef5d admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.121276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bcdaa1eb-5f93-41f3-b79e-d1cd3d6eef5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:57.194509 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fb3a3-e5d2-4092-9626-c666f680f7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:57.195502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fb3a3-e5d2-4092-9626-c666f680f7b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.195791 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fb3a3-e5d2-4092-9626-c666f680f7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.196065 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fb3a3-e5d2-4092-9626-c666f680f7b7 admin admin] PATH_INFO: `/v3/projects/87877dbc4a8a454182f279c412275476` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.197534 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7fb3a3-e5d2-4092-9626-c666f680f7b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87877dbc4a8a454182f279c412275476)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:57.199734 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7fb3a3-e5d2-4092-9626-c666f680f7b7 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:57.230349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:57.239541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:57.251257 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdaa1eb-5f93-41f3-b79e-d1cd3d6eef5d admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:57.278885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-3c7fb3a3-e5d2-4092-9626-c666f680f7b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87877dbc4a8a454182f279c412275476', 'request_id': 'req-3c7fb3a3-e5d2-4092-9626-c666f680f7b7'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:57.291661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-3c7fb3a3-e5d2-4092-9626-c666f680f7b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87877dbc4a8a454182f279c412275476', 'request_id': 'req-3c7fb3a3-e5d2-4092-9626-c666f680f7b7'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:57.301000 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-3c7fb3a3-e5d2-4092-9626-c666f680f7b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87877dbc4a8a454182f279c412275476', 'request_id': 'req-3c7fb3a3-e5d2-4092-9626-c666f680f7b7'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:57.319586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-3c7fb3a3-e5d2-4092-9626-c666f680f7b7 admin admin] The token cache is being invalidate because project 87877dbc4a8a454182f279c412275476 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:57.320977 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-3c7fb3a3-e5d2-4092-9626-c666f680f7b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:57.321981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-3c7fb3a3-e5d2-4092-9626-c666f680f7b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:57.324324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1268/2557] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:57 2022] DELETE /identity/v3/projects/87877dbc4a8a454182f279c412275476 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:57.330845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6459d4-6803-438a-bcb4-172bfa33410b None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.331194 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6459d4-6803-438a-bcb4-172bfa33410b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.331505 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6459d4-6803-438a-bcb4-172bfa33410b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.354295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe41359-2905-423c-9c93-1cdaee33e814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:57.355331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe41359-2905-423c-9c93-1cdaee33e814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.355781 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe41359-2905-423c-9c93-1cdaee33e814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.356225 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe41359-2905-423c-9c93-1cdaee33e814 admin admin] PATH_INFO: `/v3/projects/3243325520f34e83afdc5e605fcab9f4` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.358997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe41359-2905-423c-9c93-1cdaee33e814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3243325520f34e83afdc5e605fcab9f4)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:57.360717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-9f6459d4-6803-438a-bcb4-172bfa33410b None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:57.362574 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe41359-2905-423c-9c93-1cdaee33e814 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:57.362847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-9f6459d4-6803-438a-bcb4-172bfa33410b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:57.376771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1269/2558] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:57.389161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72653766-85e3-4b22-9523-c8dbcb7d0403 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.389612 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72653766-85e3-4b22-9523-c8dbcb7d0403 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.389821 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72653766-85e3-4b22-9523-c8dbcb7d0403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.422742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-72653766-85e3-4b22-9523-c8dbcb7d0403 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:57.424753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-72653766-85e3-4b22-9523-c8dbcb7d0403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:57.443839 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-fbe41359-2905-423c-9c93-1cdaee33e814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3243325520f34e83afdc5e605fcab9f4', 'request_id': 'req-fbe41359-2905-423c-9c93-1cdaee33e814'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:57.452033 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-fbe41359-2905-423c-9c93-1cdaee33e814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3243325520f34e83afdc5e605fcab9f4', 'request_id': 'req-fbe41359-2905-423c-9c93-1cdaee33e814'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:57.466207 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-fbe41359-2905-423c-9c93-1cdaee33e814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3243325520f34e83afdc5e605fcab9f4', 'request_id': 'req-fbe41359-2905-423c-9c93-1cdaee33e814'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:57.490705 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-fbe41359-2905-423c-9c93-1cdaee33e814 admin admin] The token cache is being invalidate because project 3243325520f34e83afdc5e605fcab9f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:57.491341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-fbe41359-2905-423c-9c93-1cdaee33e814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:57.492576 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-fbe41359-2905-423c-9c93-1cdaee33e814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:57.495387 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1290/2559] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:57 2022] DELETE /identity/v3/projects/3243325520f34e83afdc5e605fcab9f4 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:57.503176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576fdcb-9d10-4a5e-8bfe-574254c48d5e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.503629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576fdcb-9d10-4a5e-8bfe-574254c48d5e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.504104 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576fdcb-9d10-4a5e-8bfe-574254c48d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.541137 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e576fdcb-9d10-4a5e-8bfe-574254c48d5e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:57.543404 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e576fdcb-9d10-4a5e-8bfe-574254c48d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:57.549199 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1270/2560] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:57.555940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72653766-85e3-4b22-9523-c8dbcb7d0403 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.558420 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1291/2561] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:57.572535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc1039-dd47-4e4d-9d89-c0001f468956 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.572917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc1039-dd47-4e4d-9d89-c0001f468956 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.573486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc1039-dd47-4e4d-9d89-c0001f468956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.575238 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72653766-85e3-4b22-9523-c8dbcb7d0403 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:57.584581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf45fe-3921-4e71-b1e9-03430e2302f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:57.585711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf45fe-3921-4e71-b1e9-03430e2302f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.586212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf45fe-3921-4e71-b1e9-03430e2302f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.586706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf45fe-3921-4e71-b1e9-03430e2302f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.587483 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf45fe-3921-4e71-b1e9-03430e2302f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:32:57.587854 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf45fe-3921-4e71-b1e9-03430e2302f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:57.591062 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf45fe-3921-4e71-b1e9-03430e2302f4 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:57.605083 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-4efc1039-dd47-4e4d-9d89-c0001f468956 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:57.606611 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4efc1039-dd47-4e4d-9d89-c0001f468956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:57.614401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1271/2562] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:32:57.619307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1292/2563] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:57.621738 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf45fe-3921-4e71-b1e9-03430e2302f4 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.626784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc1039-dd47-4e4d-9d89-c0001f468956 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.637984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf45fe-3921-4e71-b1e9-03430e2302f4 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:57.640609 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc1039-dd47-4e4d-9d89-c0001f468956 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:57.644058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbab3d9-e928-4c2b-a0b4-def544d20c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:57.644830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbab3d9-e928-4c2b-a0b4-def544d20c40 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.645314 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbab3d9-e928-4c2b-a0b4-def544d20c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.645655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbab3d9-e928-4c2b-a0b4-def544d20c40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.646535 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbab3d9-e928-4c2b-a0b4-def544d20c40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:57.648805 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbab3d9-e928-4c2b-a0b4-def544d20c40 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:57.649524 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89bb58-389e-4f4e-a16b-74e2c386a216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:57.650553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89bb58-389e-4f4e-a16b-74e2c386a216 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.651052 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89bb58-389e-4f4e-a16b-74e2c386a216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.651551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89bb58-389e-4f4e-a16b-74e2c386a216 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.652793 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89bb58-389e-4f4e-a16b-74e2c386a216 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:32:57.653547 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89bb58-389e-4f4e-a16b-74e2c386a216 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:57.657285 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89bb58-389e-4f4e-a16b-74e2c386a216 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:57.686099 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1293/2564] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:32:57.687779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1272/2565] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:57 2022] POST /identity/v3/projects => generated 369 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:57.688308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89bb58-389e-4f4e-a16b-74e2c386a216 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.692196 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbab3d9-e928-4c2b-a0b4-def544d20c40 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.701657 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89bb58-389e-4f4e-a16b-74e2c386a216 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:57.790233 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7841ec-2de5-45a1-9448-4ec1de392922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:57.791266 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7841ec-2de5-45a1-9448-4ec1de392922 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.791699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7841ec-2de5-45a1-9448-4ec1de392922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.792113 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7841ec-2de5-45a1-9448-4ec1de392922 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.793237 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7841ec-2de5-45a1-9448-4ec1de392922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:57.797279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7841ec-2de5-45a1-9448-4ec1de392922 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:57.814866 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbab3d9-e928-4c2b-a0b4-def544d20c40 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:57.822776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfa0bf-3b3e-4b21-81a0-ec8e227181cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:57.823572 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfa0bf-3b3e-4b21-81a0-ec8e227181cf admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.823957 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfa0bf-3b3e-4b21-81a0-ec8e227181cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.824296 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfa0bf-3b3e-4b21-81a0-ec8e227181cf admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.825010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfa0bf-3b3e-4b21-81a0-ec8e227181cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:57.828020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfa0bf-3b3e-4b21-81a0-ec8e227181cf admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:57.849162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:57.876001 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1273/2566] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:57 2022] POST /identity/v3/users => generated 285 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:57.882556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfa0bf-3b3e-4b21-81a0-ec8e227181cf admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.901695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfa0bf-3b3e-4b21-81a0-ec8e227181cf admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:57.912146 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef724e98-de9c-474d-8af5-71677c028c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:57.913385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef724e98-de9c-474d-8af5-71677c028c40 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.913773 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef724e98-de9c-474d-8af5-71677c028c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.914240 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef724e98-de9c-474d-8af5-71677c028c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.915465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef724e98-de9c-474d-8af5-71677c028c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:57.918554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef724e98-de9c-474d-8af5-71677c028c40 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:57.939907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1274/2567] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:32:57.944500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef724e98-de9c-474d-8af5-71677c028c40 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.963296 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef724e98-de9c-474d-8af5-71677c028c40 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:57.973045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8efc6-af99-48b1-b8c9-4c5c8eaa4589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:57.974078 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8efc6-af99-48b1-b8c9-4c5c8eaa4589 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.974324 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:57.974564 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8efc6-af99-48b1-b8c9-4c5c8eaa4589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.975165 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8efc6-af99-48b1-b8c9-4c5c8eaa4589 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.976718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8efc6-af99-48b1-b8c9-4c5c8eaa4589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:57.979345 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8efc6-af99-48b1-b8c9-4c5c8eaa4589 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.002885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1275/2568] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:58.009468 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8efc6-af99-48b1-b8c9-4c5c8eaa4589 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:58.013195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1294/2569] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:57 2022] POST /identity/v3/projects => generated 393 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:58.022707 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7841ec-2de5-45a1-9448-4ec1de392922 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:58.031609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8efc6-af99-48b1-b8c9-4c5c8eaa4589 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:58.123518 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb200df-af7e-457a-8c55-f43cf54d43d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:58.124754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb200df-af7e-457a-8c55-f43cf54d43d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.125906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb200df-af7e-457a-8c55-f43cf54d43d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.125906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb200df-af7e-457a-8c55-f43cf54d43d2 admin admin] PATH_INFO: `/v3/projects/faea4df22653419d8b6037254dc070f0/users/02f9f109650f484db0357ec6a2154a41/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.148605 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7841ec-2de5-45a1-9448-4ec1de392922 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:58.153034 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb200df-af7e-457a-8c55-f43cf54d43d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faea4df22653419d8b6037254dc070f0, user_id=02f9f109650f484db0357ec6a2154a41, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:58.157910 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb200df-af7e-457a-8c55-f43cf54d43d2 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.158258 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56255fa-af36-4f40-8b7b-990241e55d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:58.158929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56255fa-af36-4f40-8b7b-990241e55d8e admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.159213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56255fa-af36-4f40-8b7b-990241e55d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.159485 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56255fa-af36-4f40-8b7b-990241e55d8e admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.160294 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56255fa-af36-4f40-8b7b-990241e55d8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:58.162515 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56255fa-af36-4f40-8b7b-990241e55d8e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.179740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1276/2570] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:58 2022] PUT /identity/v3/projects/faea4df22653419d8b6037254dc070f0/users/02f9f109650f484db0357ec6a2154a41/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:58.201198 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1295/2571] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:58 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:58.205407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cade7-ccc5-4ad3-9ad8-ee06845b186a None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.205407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cade7-ccc5-4ad3-9ad8-ee06845b186a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.205407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cade7-ccc5-4ad3-9ad8-ee06845b186a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.208873 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56255fa-af36-4f40-8b7b-990241e55d8e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:58.230689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56255fa-af36-4f40-8b7b-990241e55d8e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:58.256039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-4f8cade7-ccc5-4ad3-9ad8-ee06845b186a None None] MFA Rules not processed for user `02f9f109650f484db0357ec6a2154a41`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:58.257328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4f8cade7-ccc5-4ad3-9ad8-ee06845b186a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:58.348078 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb9687-fd6b-454e-a33f-b9969241cec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:58.348975 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb9687-fd6b-454e-a33f-b9969241cec5 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.349299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb9687-fd6b-454e-a33f-b9969241cec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.349622 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb9687-fd6b-454e-a33f-b9969241cec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.350498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb9687-fd6b-454e-a33f-b9969241cec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:58.354488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb9687-fd6b-454e-a33f-b9969241cec5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.375985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1277/2572] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:58.379244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1296/2573] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:32:58.386338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cade7-ccc5-4ad3-9ad8-ee06845b186a None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:58.399825 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb9687-fd6b-454e-a33f-b9969241cec5 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:58.402665 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-31eb9687-fd6b-454e-a33f-b9969241cec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:58.405501 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cade7-ccc5-4ad3-9ad8-ee06845b186a None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:58.414488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123286f7-a76b-41d1-8c77-338380dd678d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:58.415776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123286f7-a76b-41d1-8c77-338380dd678d admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.416395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123286f7-a76b-41d1-8c77-338380dd678d admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.416970 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123286f7-a76b-41d1-8c77-338380dd678d admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.418283 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123286f7-a76b-41d1-8c77-338380dd678d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:58.421928 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123286f7-a76b-41d1-8c77-338380dd678d admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.445514 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1278/2574] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:58.450426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123286f7-a76b-41d1-8c77-338380dd678d admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:58.470034 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123286f7-a76b-41d1-8c77-338380dd678d admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:58.478903 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8ac2d-3fec-44d4-becb-8b62f4741818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:58.480073 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8ac2d-3fec-44d4-becb-8b62f4741818 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.480560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8ac2d-3fec-44d4-becb-8b62f4741818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.481045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8ac2d-3fec-44d4-becb-8b62f4741818 admin admin] PATH_INFO: `/v3/projects/e60b19cfe8344d169678943e4e6bd87c/users/2547bdb3ef1b49c285af0a64a565a816/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.511530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb9687-fd6b-454e-a33f-b9969241cec5 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:58.517473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325371af-3348-4b16-b542-3fb1468ba397 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2211e237fe554a82baf129e440a0fb43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:58.518500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325371af-3348-4b16-b542-3fb1468ba397 service glance] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.518960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325371af-3348-4b16-b542-3fb1468ba397 service glance] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.519363 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325371af-3348-4b16-b542-3fb1468ba397 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.525263 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8ac2d-3fec-44d4-becb-8b62f4741818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e60b19cfe8344d169678943e4e6bd87c, user_id=2547bdb3ef1b49c285af0a64a565a816, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:58.532284 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8ac2d-3fec-44d4-becb-8b62f4741818 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.535098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325371af-3348-4b16-b542-3fb1468ba397 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:58.537808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325371af-3348-4b16-b542-3fb1468ba397 service glance] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.558144 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1297/2575] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:32:58.558423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1279/2576] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:58 2022] PUT /identity/v3/projects/e60b19cfe8344d169678943e4e6bd87c/users/2547bdb3ef1b49c285af0a64a565a816/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:58.570162 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b89ec6-c557-4732-9ea7-cb71644b9d1d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.570642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b89ec6-c557-4732-9ea7-cb71644b9d1d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.571140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b89ec6-c557-4732-9ea7-cb71644b9d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.621334 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a2b89ec6-c557-4732-9ea7-cb71644b9d1d None None] MFA Rules not processed for user `2547bdb3ef1b49c285af0a64a565a816`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:58.623506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a2b89ec6-c557-4732-9ea7-cb71644b9d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:58.641232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:58.777427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1298/2577] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:58.784314 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8ac2d-3fec-44d4-becb-8b62f4741818 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:58.810684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:58.923287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8ac2d-3fec-44d4-becb-8b62f4741818 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:58.930452 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c53083-0df6-4168-b212-5016f97af175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:58.931137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c53083-0df6-4168-b212-5016f97af175 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.931584 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c53083-0df6-4168-b212-5016f97af175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.933265 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c53083-0df6-4168-b212-5016f97af175 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.935047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c53083-0df6-4168-b212-5016f97af175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:58.940973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c53083-0df6-4168-b212-5016f97af175 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.996511 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1280/2578] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:58 2022] POST /identity/v3/projects => generated 393 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:59.005445 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b89ec6-c557-4732-9ea7-cb71644b9d1d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:59.027613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b89ec6-c557-4732-9ea7-cb71644b9d1d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:59.135498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89657432-bca9-4841-a780-fe142ddfc36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:59.137079 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89657432-bca9-4841-a780-fe142ddfc36b admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:59.137735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89657432-bca9-4841-a780-fe142ddfc36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:59.138093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89657432-bca9-4841-a780-fe142ddfc36b admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:59.139311 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89657432-bca9-4841-a780-fe142ddfc36b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:59.144179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89657432-bca9-4841-a780-fe142ddfc36b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:59.192304 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1299/2579] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:59 2022] POST /identity/v3/users => generated 303 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:59.198817 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c53083-0df6-4168-b212-5016f97af175 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:59.220198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c53083-0df6-4168-b212-5016f97af175 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:59.228557 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790adfcf-78b1-4880-8022-13e15203242e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:59.229693 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790adfcf-78b1-4880-8022-13e15203242e admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:59.230195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790adfcf-78b1-4880-8022-13e15203242e admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:59.230683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790adfcf-78b1-4880-8022-13e15203242e admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:59.231882 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790adfcf-78b1-4880-8022-13e15203242e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:59.235720 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790adfcf-78b1-4880-8022-13e15203242e admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:59.259218 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1281/2580] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:59.267438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89657432-bca9-4841-a780-fe142ddfc36b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:59.288352 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89657432-bca9-4841-a780-fe142ddfc36b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:59.294914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770dee2-809f-4f14-bfc2-668079548e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:59.295713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770dee2-809f-4f14-bfc2-668079548e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:59.297643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770dee2-809f-4f14-bfc2-668079548e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:59.298081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770dee2-809f-4f14-bfc2-668079548e94 admin admin] PATH_INFO: `/v3/projects/2c65306a22084d2dacf9f92a3ebda5d8/users/908b2f40b39948cca6492842dae7b1bb/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:59.320802 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2770dee2-809f-4f14-bfc2-668079548e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c65306a22084d2dacf9f92a3ebda5d8, user_id=908b2f40b39948cca6492842dae7b1bb, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:59.326169 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2770dee2-809f-4f14-bfc2-668079548e94 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:59.368436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1300/2581] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:59 2022] PUT /identity/v3/projects/2c65306a22084d2dacf9f92a3ebda5d8/users/908b2f40b39948cca6492842dae7b1bb/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:59.371784 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790adfcf-78b1-4880-8022-13e15203242e admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:59.390894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790adfcf-78b1-4880-8022-13e15203242e admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:59.506763 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6acaf6-340d-4649-9ee4-675b5754f037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:59.507202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6acaf6-340d-4649-9ee4-675b5754f037 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:59.507626 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6acaf6-340d-4649-9ee4-675b5754f037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:59.508068 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6acaf6-340d-4649-9ee4-675b5754f037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:59.508826 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6acaf6-340d-4649-9ee4-675b5754f037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:59.511174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6acaf6-340d-4649-9ee4-675b5754f037 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:59.527426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1282/2582] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:59.534772 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770dee2-809f-4f14-bfc2-668079548e94 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:59.551385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770dee2-809f-4f14-bfc2-668079548e94 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:59.559901 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07714f-0e67-41e4-afbf-da503d3cb7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:59.561093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07714f-0e67-41e4-afbf-da503d3cb7eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:59.561587 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07714f-0e67-41e4-afbf-da503d3cb7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:59.562065 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07714f-0e67-41e4-afbf-da503d3cb7eb admin admin] PATH_INFO: `/v3/projects/2c65306a22084d2dacf9f92a3ebda5d8/users/908b2f40b39948cca6492842dae7b1bb/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:59.565718 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07714f-0e67-41e4-afbf-da503d3cb7eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c65306a22084d2dacf9f92a3ebda5d8, user_id=908b2f40b39948cca6492842dae7b1bb, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:59.571259 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07714f-0e67-41e4-afbf-da503d3cb7eb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:59.591006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1301/2583] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:59 2022] PUT /identity/v3/projects/2c65306a22084d2dacf9f92a3ebda5d8/users/908b2f40b39948cca6492842dae7b1bb/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:59.615175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713164c-c620-4bf7-8068-d820fe8f0d8b None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:59.615519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713164c-c620-4bf7-8068-d820fe8f0d8b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:59.615848 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713164c-c620-4bf7-8068-d820fe8f0d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:59.665385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-d713164c-c620-4bf7-8068-d820fe8f0d8b None None] MFA Rules not processed for user `908b2f40b39948cca6492842dae7b1bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:59.667044 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d713164c-c620-4bf7-8068-d820fe8f0d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:59.774112 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1283/2584] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:32:59 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:59.793677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07714f-0e67-41e4-afbf-da503d3cb7eb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:59.797565 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5b07714f-0e67-41e4-afbf-da503d3cb7eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:59.915206 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07714f-0e67-41e4-afbf-da503d3cb7eb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:59.922537 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360839e3-8a7c-4232-9ec1-f251ed54efb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:59.923974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360839e3-8a7c-4232-9ec1-f251ed54efb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:59.924437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360839e3-8a7c-4232-9ec1-f251ed54efb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:59.924929 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360839e3-8a7c-4232-9ec1-f251ed54efb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:59.941899 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360839e3-8a7c-4232-9ec1-f251ed54efb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:59.944567 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360839e3-8a7c-4232-9ec1-f251ed54efb3 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:59.965088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1302/2585] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:59 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:00.054943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713164c-c620-4bf7-8068-d820fe8f0d8b None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:00.057786 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d713164c-c620-4bf7-8068-d820fe8f0d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:00.081707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713164c-c620-4bf7-8068-d820fe8f0d8b None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:00.086273 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010aafb-7e3a-4b05-9614-e2eaf9a9aa9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:00.086978 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010aafb-7e3a-4b05-9614-e2eaf9a9aa9c service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:00.087263 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010aafb-7e3a-4b05-9614-e2eaf9a9aa9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:00.087623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010aafb-7e3a-4b05-9614-e2eaf9a9aa9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:00.093181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:00.182653 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010aafb-7e3a-4b05-9614-e2eaf9a9aa9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:00.185482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010aafb-7e3a-4b05-9614-e2eaf9a9aa9c service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:00.203901 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1284/2586] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:04.170051 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360839e3-8a7c-4232-9ec1-f251ed54efb3 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:04.177776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:04.293892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360839e3-8a7c-4232-9ec1-f251ed54efb3 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:04.303090 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec75291-4afe-449e-a7f9-b3503a0be8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:04.304443 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec75291-4afe-449e-a7f9-b3503a0be8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:04.305192 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec75291-4afe-449e-a7f9-b3503a0be8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:04.305804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec75291-4afe-449e-a7f9-b3503a0be8fd admin admin] PATH_INFO: `/v3/users/2547bdb3ef1b49c285af0a64a565a816` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:04.308401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec75291-4afe-449e-a7f9-b3503a0be8fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2547bdb3ef1b49c285af0a64a565a816)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:04.314475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec75291-4afe-449e-a7f9-b3503a0be8fd admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:04.408158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-aec75291-4afe-449e-a7f9-b3503a0be8fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2547bdb3ef1b49c285af0a64a565a816'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:04.434433 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-aec75291-4afe-449e-a7f9-b3503a0be8fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2547bdb3ef1b49c285af0a64a565a816'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:04.446257 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-aec75291-4afe-449e-a7f9-b3503a0be8fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2547bdb3ef1b49c285af0a64a565a816'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:04.446980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-aec75291-4afe-449e-a7f9-b3503a0be8fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2547bdb3ef1b49c285af0a64a565a816'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:04.456289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-aec75291-4afe-449e-a7f9-b3503a0be8fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2547bdb3ef1b49c285af0a64a565a816'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:04.469303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1303/2587] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:04 2022] DELETE /identity/v3/users/2547bdb3ef1b49c285af0a64a565a816 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:04.471922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85dcb5-8e5e-44d7-a10d-10c8a78ccb38 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:04.472779 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85dcb5-8e5e-44d7-a10d-10c8a78ccb38 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:04.473150 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85dcb5-8e5e-44d7-a10d-10c8a78ccb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:04.496749 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-eb85dcb5-8e5e-44d7-a10d-10c8a78ccb38 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:04.498152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-eb85dcb5-8e5e-44d7-a10d-10c8a78ccb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:04.572234 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1285/2588] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:04.585782 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec75291-4afe-449e-a7f9-b3503a0be8fd admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:04.587261 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-aec75291-4afe-449e-a7f9-b3503a0be8fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:04.653630 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec75291-4afe-449e-a7f9-b3503a0be8fd admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:04.657954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b540601-c9dc-4fc2-9467-884896af049d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:04.658569 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b540601-c9dc-4fc2-9467-884896af049d service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:04.658849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b540601-c9dc-4fc2-9467-884896af049d service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:04.659148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b540601-c9dc-4fc2-9467-884896af049d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:04.669188 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b540601-c9dc-4fc2-9467-884896af049d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:04.671157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b540601-c9dc-4fc2-9467-884896af049d service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:04.682533 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1304/2589] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:05.059936 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85dcb5-8e5e-44d7-a10d-10c8a78ccb38 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:05.062003 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-eb85dcb5-8e5e-44d7-a10d-10c8a78ccb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:05.075296 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85dcb5-8e5e-44d7-a10d-10c8a78ccb38 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:05.081361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58265051-d055-4dff-b7fa-332304da307f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:05.082064 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58265051-d055-4dff-b7fa-332304da307f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:05.082451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58265051-d055-4dff-b7fa-332304da307f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:05.082795 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58265051-d055-4dff-b7fa-332304da307f admin admin] PATH_INFO: `/v3/projects/e60b19cfe8344d169678943e4e6bd87c` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:05.083757 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58265051-d055-4dff-b7fa-332304da307f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e60b19cfe8344d169678943e4e6bd87c)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:05.085729 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58265051-d055-4dff-b7fa-332304da307f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:05.144286 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-58265051-d055-4dff-b7fa-332304da307f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e60b19cfe8344d169678943e4e6bd87c', 'request_id': 'req-58265051-d055-4dff-b7fa-332304da307f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:05.151493 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-58265051-d055-4dff-b7fa-332304da307f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e60b19cfe8344d169678943e4e6bd87c', 'request_id': 'req-58265051-d055-4dff-b7fa-332304da307f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:05.158789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-58265051-d055-4dff-b7fa-332304da307f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e60b19cfe8344d169678943e4e6bd87c', 'request_id': 'req-58265051-d055-4dff-b7fa-332304da307f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:05.172371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-58265051-d055-4dff-b7fa-332304da307f admin admin] The token cache is being invalidate because project e60b19cfe8344d169678943e4e6bd87c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:05.172704 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-58265051-d055-4dff-b7fa-332304da307f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:05.173364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-58265051-d055-4dff-b7fa-332304da307f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:05.175221 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1286/2590] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:05 2022] DELETE /identity/v3/projects/e60b19cfe8344d169678943e4e6bd87c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:05.179597 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b540601-c9dc-4fc2-9467-884896af049d service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:05.191219 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b540601-c9dc-4fc2-9467-884896af049d service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:05.219156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:05.258763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd327b-f012-4755-a1e5-1bff9f5cb57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:05.259463 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd327b-f012-4755-a1e5-1bff9f5cb57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:05.259711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd327b-f012-4755-a1e5-1bff9f5cb57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:05.259932 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd327b-f012-4755-a1e5-1bff9f5cb57f admin admin] PATH_INFO: `/v3/users/908b2f40b39948cca6492842dae7b1bb` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:05.262063 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd327b-f012-4755-a1e5-1bff9f5cb57f admin admin] RBAC: Authorizing `identity:delete_user(user_id=908b2f40b39948cca6492842dae7b1bb)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:05.263985 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd327b-f012-4755-a1e5-1bff9f5cb57f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:05.323267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-33cd327b-f012-4755-a1e5-1bff9f5cb57f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '908b2f40b39948cca6492842dae7b1bb'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:05.343044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-33cd327b-f012-4755-a1e5-1bff9f5cb57f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '908b2f40b39948cca6492842dae7b1bb'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:05.354790 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-33cd327b-f012-4755-a1e5-1bff9f5cb57f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '908b2f40b39948cca6492842dae7b1bb'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:05.355430 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-33cd327b-f012-4755-a1e5-1bff9f5cb57f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '908b2f40b39948cca6492842dae7b1bb'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:05.367093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-33cd327b-f012-4755-a1e5-1bff9f5cb57f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '908b2f40b39948cca6492842dae7b1bb'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:05.382757 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1305/2591] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:05 2022] DELETE /identity/v3/users/908b2f40b39948cca6492842dae7b1bb => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:05.815933 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58265051-d055-4dff-b7fa-332304da307f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:05.817915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-58265051-d055-4dff-b7fa-332304da307f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:05.833607 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58265051-d055-4dff-b7fa-332304da307f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:05.876837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:05.928788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b60d7-c8e0-4e93-a2ba-23dbccd160a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:05.929479 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b60d7-c8e0-4e93-a2ba-23dbccd160a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:05.929754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b60d7-c8e0-4e93-a2ba-23dbccd160a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:05.930024 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b60d7-c8e0-4e93-a2ba-23dbccd160a6 admin admin] PATH_INFO: `/v3/projects/2c65306a22084d2dacf9f92a3ebda5d8` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:05.931204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b60d7-c8e0-4e93-a2ba-23dbccd160a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c65306a22084d2dacf9f92a3ebda5d8)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:05.933189 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b60d7-c8e0-4e93-a2ba-23dbccd160a6 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:06.017960 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-914b60d7-c8e0-4e93-a2ba-23dbccd160a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c65306a22084d2dacf9f92a3ebda5d8', 'request_id': 'req-914b60d7-c8e0-4e93-a2ba-23dbccd160a6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:06.026574 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-914b60d7-c8e0-4e93-a2ba-23dbccd160a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c65306a22084d2dacf9f92a3ebda5d8', 'request_id': 'req-914b60d7-c8e0-4e93-a2ba-23dbccd160a6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:06.036186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-914b60d7-c8e0-4e93-a2ba-23dbccd160a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c65306a22084d2dacf9f92a3ebda5d8', 'request_id': 'req-914b60d7-c8e0-4e93-a2ba-23dbccd160a6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:06.065747 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-914b60d7-c8e0-4e93-a2ba-23dbccd160a6 admin admin] The token cache is being invalidate because project 2c65306a22084d2dacf9f92a3ebda5d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:06.066299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-914b60d7-c8e0-4e93-a2ba-23dbccd160a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:06.067658 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-914b60d7-c8e0-4e93-a2ba-23dbccd160a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:06.070492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1287/2592] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:05 2022] DELETE /identity/v3/projects/2c65306a22084d2dacf9f92a3ebda5d8 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:06.077725 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00055f72-c65e-47cf-9534-15458b0fbe0b None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:06.078535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00055f72-c65e-47cf-9534-15458b0fbe0b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:06.079112 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00055f72-c65e-47cf-9534-15458b0fbe0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:06.103916 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-00055f72-c65e-47cf-9534-15458b0fbe0b None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:06.105187 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-00055f72-c65e-47cf-9534-15458b0fbe0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:06.176401 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1306/2593] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:06.186694 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe43a23-fdff-4c2d-8024-2e9f1b5e657f None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:06.187376 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe43a23-fdff-4c2d-8024-2e9f1b5e657f None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:06.187864 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe43a23-fdff-4c2d-8024-2e9f1b5e657f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:06.213236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-dfe43a23-fdff-4c2d-8024-2e9f1b5e657f None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:06.213494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-dfe43a23-fdff-4c2d-8024-2e9f1b5e657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:06.222316 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1288/2594] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:06.226738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00055f72-c65e-47cf-9534-15458b0fbe0b None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:06.237571 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00055f72-c65e-47cf-9534-15458b0fbe0b None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:06.242883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab2689-fd68-416a-9f54-a99c711569e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:06.243595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab2689-fd68-416a-9f54-a99c711569e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:06.243964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab2689-fd68-416a-9f54-a99c711569e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:06.244311 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab2689-fd68-416a-9f54-a99c711569e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:06.245167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab2689-fd68-416a-9f54-a99c711569e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:06.245485 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab2689-fd68-416a-9f54-a99c711569e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:06.247406 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab2689-fd68-416a-9f54-a99c711569e3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:06.263169 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1307/2595] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:33:06.268701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe43a23-fdff-4c2d-8024-2e9f1b5e657f None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:06.280567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe43a23-fdff-4c2d-8024-2e9f1b5e657f None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:06.286784 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40109d5e-be9d-4dfd-8799-308a9db1d728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:06.287588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40109d5e-be9d-4dfd-8799-308a9db1d728 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:06.287882 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40109d5e-be9d-4dfd-8799-308a9db1d728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:06.288265 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40109d5e-be9d-4dfd-8799-308a9db1d728 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:06.288916 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40109d5e-be9d-4dfd-8799-308a9db1d728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:06.290976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40109d5e-be9d-4dfd-8799-308a9db1d728 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:06.333500 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1289/2596] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:06 2022] POST /identity/v3/projects => generated 387 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:06.338007 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab2689-fd68-416a-9f54-a99c711569e3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:06.351942 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab2689-fd68-416a-9f54-a99c711569e3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:06.410638 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:06.453622 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd6c6b-ec32-4701-a214-be32d5978adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:06.454378 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd6c6b-ec32-4701-a214-be32d5978adb admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:06.454661 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd6c6b-ec32-4701-a214-be32d5978adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:06.454935 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd6c6b-ec32-4701-a214-be32d5978adb admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:06.455586 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd6c6b-ec32-4701-a214-be32d5978adb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:06.457655 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd6c6b-ec32-4701-a214-be32d5978adb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:06.493145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1308/2597] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:06 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:06.496859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40109d5e-be9d-4dfd-8799-308a9db1d728 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:06.515058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40109d5e-be9d-4dfd-8799-308a9db1d728 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:06.520652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d1beb-2bfd-4fb6-a577-31aed03f0b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:06.521615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d1beb-2bfd-4fb6-a577-31aed03f0b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:06.521991 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d1beb-2bfd-4fb6-a577-31aed03f0b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:06.522357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d1beb-2bfd-4fb6-a577-31aed03f0b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:06.523282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d1beb-2bfd-4fb6-a577-31aed03f0b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:06.525260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d1beb-2bfd-4fb6-a577-31aed03f0b18 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:06.547551 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1290/2598] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:06.550105 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd6c6b-ec32-4701-a214-be32d5978adb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:06.568667 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd6c6b-ec32-4701-a214-be32d5978adb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:06.573855 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec433cf-0d97-4c7c-8ccf-1458f003c293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:06.574489 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec433cf-0d97-4c7c-8ccf-1458f003c293 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:06.574767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec433cf-0d97-4c7c-8ccf-1458f003c293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:06.575049 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec433cf-0d97-4c7c-8ccf-1458f003c293 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:06.575730 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec433cf-0d97-4c7c-8ccf-1458f003c293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:06.578695 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec433cf-0d97-4c7c-8ccf-1458f003c293 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:06.590025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1309/2599] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:06.596127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d1beb-2bfd-4fb6-a577-31aed03f0b18 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:06.607396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d1beb-2bfd-4fb6-a577-31aed03f0b18 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:06.614009 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938be48-caeb-48ea-a1c8-c1edf448b43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:06.614646 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938be48-caeb-48ea-a1c8-c1edf448b43e admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:06.615176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938be48-caeb-48ea-a1c8-c1edf448b43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:06.615367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938be48-caeb-48ea-a1c8-c1edf448b43e admin admin] PATH_INFO: `/v3/projects/9eb778185cf1463095b9699d8cbe9d8d/users/ca57f9a0d7054233b904c60705ad1c39/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:06.636762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938be48-caeb-48ea-a1c8-c1edf448b43e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb778185cf1463095b9699d8cbe9d8d, user_id=ca57f9a0d7054233b904c60705ad1c39, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:06.639704 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938be48-caeb-48ea-a1c8-c1edf448b43e admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:06.656310 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1291/2600] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:06 2022] PUT /identity/v3/projects/9eb778185cf1463095b9699d8cbe9d8d/users/ca57f9a0d7054233b904c60705ad1c39/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:06.668117 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b4b77-584e-4cba-a1ea-27d0920324f9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:06.668406 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b4b77-584e-4cba-a1ea-27d0920324f9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:06.668845 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b4b77-584e-4cba-a1ea-27d0920324f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:06.730084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-850b4b77-584e-4cba-a1ea-27d0920324f9 None None] MFA Rules not processed for user `ca57f9a0d7054233b904c60705ad1c39`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:06.734405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-850b4b77-584e-4cba-a1ea-27d0920324f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:06.899208 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1310/2601] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:06.906639 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938be48-caeb-48ea-a1c8-c1edf448b43e admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:07.032401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:07.053146 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938be48-caeb-48ea-a1c8-c1edf448b43e admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:07.064371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123d77f-6c3c-4f0f-9d2f-cd4de9817f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:07.065097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123d77f-6c3c-4f0f-9d2f-cd4de9817f4f admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:07.065367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123d77f-6c3c-4f0f-9d2f-cd4de9817f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:07.065678 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123d77f-6c3c-4f0f-9d2f-cd4de9817f4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:07.066369 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123d77f-6c3c-4f0f-9d2f-cd4de9817f4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:07.068560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123d77f-6c3c-4f0f-9d2f-cd4de9817f4f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:07.157687 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1292/2602] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:06 2022] POST /identity/v3/projects => generated 383 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:07.162985 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b4b77-584e-4cba-a1ea-27d0920324f9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:07.183314 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b4b77-584e-4cba-a1ea-27d0920324f9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:07.350191 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8883d-9154-4b0c-b5d3-57f304345dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:07.351167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8883d-9154-4b0c-b5d3-57f304345dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:07.351167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8883d-9154-4b0c-b5d3-57f304345dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:07.351475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8883d-9154-4b0c-b5d3-57f304345dd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:07.352136 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8883d-9154-4b0c-b5d3-57f304345dd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:07.355494 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8883d-9154-4b0c-b5d3-57f304345dd7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:07.394288 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1311/2603] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:07 2022] POST /identity/v3/users => generated 298 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:07.397438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123d77f-6c3c-4f0f-9d2f-cd4de9817f4f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:07.419496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123d77f-6c3c-4f0f-9d2f-cd4de9817f4f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:07.430292 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5e81d-aa5a-43c8-9dc5-51a7cac51051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:07.431605 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5e81d-aa5a-43c8-9dc5-51a7cac51051 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:07.432224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5e81d-aa5a-43c8-9dc5-51a7cac51051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:07.432997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5e81d-aa5a-43c8-9dc5-51a7cac51051 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:07.434545 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5e81d-aa5a-43c8-9dc5-51a7cac51051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:07.438710 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5e81d-aa5a-43c8-9dc5-51a7cac51051 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:07.470579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1293/2604] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:07.473232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8883d-9154-4b0c-b5d3-57f304345dd7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:07.494134 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8883d-9154-4b0c-b5d3-57f304345dd7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:07.506073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd563a3-5c45-4b85-adc3-91b77e06c116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:07.507224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd563a3-5c45-4b85-adc3-91b77e06c116 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:07.507740 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd563a3-5c45-4b85-adc3-91b77e06c116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:07.508244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd563a3-5c45-4b85-adc3-91b77e06c116 admin admin] PATH_INFO: `/v3/projects/fc4df8cc19f34d3a9a6fa2178af15ce9/users/c4e2ce1d60b743399ae7082f2aed5295/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:07.550567 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd563a3-5c45-4b85-adc3-91b77e06c116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc4df8cc19f34d3a9a6fa2178af15ce9, user_id=c4e2ce1d60b743399ae7082f2aed5295, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:07.553971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd563a3-5c45-4b85-adc3-91b77e06c116 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:07.588402 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1312/2605] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:07 2022] PUT /identity/v3/projects/fc4df8cc19f34d3a9a6fa2178af15ce9/users/c4e2ce1d60b743399ae7082f2aed5295/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:07.598167 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5e81d-aa5a-43c8-9dc5-51a7cac51051 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:07.617466 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5e81d-aa5a-43c8-9dc5-51a7cac51051 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:07.746288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ad27d-10ca-4c66-ae03-9333b05f7730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:07.747083 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ad27d-10ca-4c66-ae03-9333b05f7730 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:07.747552 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ad27d-10ca-4c66-ae03-9333b05f7730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:07.748104 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ad27d-10ca-4c66-ae03-9333b05f7730 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:07.749239 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ad27d-10ca-4c66-ae03-9333b05f7730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:07.751229 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ad27d-10ca-4c66-ae03-9333b05f7730 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:07.771351 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1294/2606] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:07.772652 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd563a3-5c45-4b85-adc3-91b77e06c116 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:07.784715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd563a3-5c45-4b85-adc3-91b77e06c116 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:07.790331 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d093d3-4881-4252-9bee-869559d60793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:07.797109 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d093d3-4881-4252-9bee-869559d60793 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:07.797426 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d093d3-4881-4252-9bee-869559d60793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:07.797690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d093d3-4881-4252-9bee-869559d60793 admin admin] PATH_INFO: `/v3/projects/fc4df8cc19f34d3a9a6fa2178af15ce9/users/c4e2ce1d60b743399ae7082f2aed5295/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:07.801423 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d093d3-4881-4252-9bee-869559d60793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc4df8cc19f34d3a9a6fa2178af15ce9, user_id=c4e2ce1d60b743399ae7082f2aed5295, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:07.803147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d093d3-4881-4252-9bee-869559d60793 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:07.806178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:07.824347 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1313/2607] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:07 2022] PUT /identity/v3/projects/fc4df8cc19f34d3a9a6fa2178af15ce9/users/c4e2ce1d60b743399ae7082f2aed5295/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:07.848659 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae9619-0fa2-41bc-afee-3adcdcc140b2 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:07.849244 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae9619-0fa2-41bc-afee-3adcdcc140b2 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:07.850015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae9619-0fa2-41bc-afee-3adcdcc140b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:07.887591 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-58ae9619-0fa2-41bc-afee-3adcdcc140b2 None None] MFA Rules not processed for user `c4e2ce1d60b743399ae7082f2aed5295`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:07.889275 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-58ae9619-0fa2-41bc-afee-3adcdcc140b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:07.991490 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1295/2608] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:08.006613 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d093d3-4881-4252-9bee-869559d60793 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:08.010807 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-17d093d3-4881-4252-9bee-869559d60793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:08.130587 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d093d3-4881-4252-9bee-869559d60793 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:08.137581 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78b56e-baa8-44db-9899-1d7c01d86571 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:08.138562 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78b56e-baa8-44db-9899-1d7c01d86571 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:08.139025 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78b56e-baa8-44db-9899-1d7c01d86571 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:08.139763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78b56e-baa8-44db-9899-1d7c01d86571 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:08.162724 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78b56e-baa8-44db-9899-1d7c01d86571 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:08.165419 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78b56e-baa8-44db-9899-1d7c01d86571 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:08.207855 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1314/2609] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:09.445165 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae9619-0fa2-41bc-afee-3adcdcc140b2 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:09.549437 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:09.573367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae9619-0fa2-41bc-afee-3adcdcc140b2 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:09.580288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25912d84-13d3-46f3-a3f5-61f73dac5902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:09.581011 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25912d84-13d3-46f3-a3f5-61f73dac5902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:09.581332 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25912d84-13d3-46f3-a3f5-61f73dac5902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:09.581601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25912d84-13d3-46f3-a3f5-61f73dac5902 admin admin] PATH_INFO: `/v3/users/ca57f9a0d7054233b904c60705ad1c39` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:09.583134 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25912d84-13d3-46f3-a3f5-61f73dac5902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca57f9a0d7054233b904c60705ad1c39)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:09.585411 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25912d84-13d3-46f3-a3f5-61f73dac5902 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:09.708683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-25912d84-13d3-46f3-a3f5-61f73dac5902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca57f9a0d7054233b904c60705ad1c39'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:09.710411 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-25912d84-13d3-46f3-a3f5-61f73dac5902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca57f9a0d7054233b904c60705ad1c39'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:09.726125 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-25912d84-13d3-46f3-a3f5-61f73dac5902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca57f9a0d7054233b904c60705ad1c39'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:09.750304 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-25912d84-13d3-46f3-a3f5-61f73dac5902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca57f9a0d7054233b904c60705ad1c39'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:09.772636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-25912d84-13d3-46f3-a3f5-61f73dac5902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca57f9a0d7054233b904c60705ad1c39'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:09.784297 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1296/2610] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:09 2022] DELETE /identity/v3/users/ca57f9a0d7054233b904c60705ad1c39 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:09.791745 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cb448-27f5-4897-8c2d-a8cfcfe2e5c4 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:09.792619 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cb448-27f5-4897-8c2d-a8cfcfe2e5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:09.792798 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cb448-27f5-4897-8c2d-a8cfcfe2e5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:09.819374 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f80cb448-27f5-4897-8c2d-a8cfcfe2e5c4 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:09.820696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f80cb448-27f5-4897-8c2d-a8cfcfe2e5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:09.947317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1315/2611] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:09.960902 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25912d84-13d3-46f3-a3f5-61f73dac5902 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:09.962478 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-25912d84-13d3-46f3-a3f5-61f73dac5902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:10.028623 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25912d84-13d3-46f3-a3f5-61f73dac5902 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:10.039055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3825f5a-cae9-4c99-a1f5-973f11e437c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:10.039954 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3825f5a-cae9-4c99-a1f5-973f11e437c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:10.040337 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3825f5a-cae9-4c99-a1f5-973f11e437c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:10.040725 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3825f5a-cae9-4c99-a1f5-973f11e437c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:10.054504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3825f5a-cae9-4c99-a1f5-973f11e437c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:10.055956 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3825f5a-cae9-4c99-a1f5-973f11e437c4 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:10.067158 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1297/2612] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:10.483696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cb448-27f5-4897-8c2d-a8cfcfe2e5c4 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:10.486357 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f80cb448-27f5-4897-8c2d-a8cfcfe2e5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:10.511416 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cb448-27f5-4897-8c2d-a8cfcfe2e5c4 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:10.525846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:10.527089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:10.527659 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:10.528220 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7 admin admin] PATH_INFO: `/v3/projects/9eb778185cf1463095b9699d8cbe9d8d` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:10.530947 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb778185cf1463095b9699d8cbe9d8d)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:10.535384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:10.625207 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb778185cf1463095b9699d8cbe9d8d', 'request_id': 'req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:10.632032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb778185cf1463095b9699d8cbe9d8d', 'request_id': 'req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:10.645593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb778185cf1463095b9699d8cbe9d8d', 'request_id': 'req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:10.661256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7 admin admin] The token cache is being invalidate because project 9eb778185cf1463095b9699d8cbe9d8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:10.661631 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:10.662427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:10.664625 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1316/2613] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:10 2022] DELETE /identity/v3/projects/9eb778185cf1463095b9699d8cbe9d8d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:10.669276 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3825f5a-cae9-4c99-a1f5-973f11e437c4 service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:10.681459 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3825f5a-cae9-4c99-a1f5-973f11e437c4 service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:10.756195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e844c16-c4b5-4854-be24-95a43fcd5550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:10.757241 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e844c16-c4b5-4854-be24-95a43fcd5550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:10.757569 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e844c16-c4b5-4854-be24-95a43fcd5550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:10.757895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e844c16-c4b5-4854-be24-95a43fcd5550 admin admin] PATH_INFO: `/v3/users/c4e2ce1d60b743399ae7082f2aed5295` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:10.759248 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e844c16-c4b5-4854-be24-95a43fcd5550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4e2ce1d60b743399ae7082f2aed5295)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:10.761436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e844c16-c4b5-4854-be24-95a43fcd5550 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:10.785363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:10.840499 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1e844c16-c4b5-4854-be24-95a43fcd5550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4e2ce1d60b743399ae7082f2aed5295'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:10.841257 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1e844c16-c4b5-4854-be24-95a43fcd5550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4e2ce1d60b743399ae7082f2aed5295'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:10.851575 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1e844c16-c4b5-4854-be24-95a43fcd5550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4e2ce1d60b743399ae7082f2aed5295'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:10.869185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1e844c16-c4b5-4854-be24-95a43fcd5550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4e2ce1d60b743399ae7082f2aed5295'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:10.880123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-1e844c16-c4b5-4854-be24-95a43fcd5550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4e2ce1d60b743399ae7082f2aed5295'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:10.891901 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1298/2614] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:10 2022] DELETE /identity/v3/users/c4e2ce1d60b743399ae7082f2aed5295 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:11.346982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:11.348715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:11.363225 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434f4f8-04b1-4d59-8ef7-45bac22ee1b7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:11.391281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:11.466170 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf9c25-0b88-4ef8-9725-4d70b5922773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:11.466716 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf9c25-0b88-4ef8-9725-4d70b5922773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:11.466991 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf9c25-0b88-4ef8-9725-4d70b5922773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:11.467745 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf9c25-0b88-4ef8-9725-4d70b5922773 admin admin] PATH_INFO: `/v3/projects/fc4df8cc19f34d3a9a6fa2178af15ce9` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:11.468895 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf9c25-0b88-4ef8-9725-4d70b5922773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc4df8cc19f34d3a9a6fa2178af15ce9)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:11.470881 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf9c25-0b88-4ef8-9725-4d70b5922773 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:11.534562 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ecf9c25-0b88-4ef8-9725-4d70b5922773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc4df8cc19f34d3a9a6fa2178af15ce9', 'request_id': 'req-0ecf9c25-0b88-4ef8-9725-4d70b5922773'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:11.540564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ecf9c25-0b88-4ef8-9725-4d70b5922773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc4df8cc19f34d3a9a6fa2178af15ce9', 'request_id': 'req-0ecf9c25-0b88-4ef8-9725-4d70b5922773'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:11.550365 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ecf9c25-0b88-4ef8-9725-4d70b5922773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc4df8cc19f34d3a9a6fa2178af15ce9', 'request_id': 'req-0ecf9c25-0b88-4ef8-9725-4d70b5922773'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:11.564764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ecf9c25-0b88-4ef8-9725-4d70b5922773 admin admin] The token cache is being invalidate because project fc4df8cc19f34d3a9a6fa2178af15ce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:11.565109 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ecf9c25-0b88-4ef8-9725-4d70b5922773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:11.565746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0ecf9c25-0b88-4ef8-9725-4d70b5922773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:11.567237 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1317/2615] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:11 2022] DELETE /identity/v3/projects/fc4df8cc19f34d3a9a6fa2178af15ce9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:11.574713 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e824ba5-c325-496d-8863-45e3132f580c None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:11.574996 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e824ba5-c325-496d-8863-45e3132f580c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:11.575274 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e824ba5-c325-496d-8863-45e3132f580c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:11.596652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-4e824ba5-c325-496d-8863-45e3132f580c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:11.597929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-4e824ba5-c325-496d-8863-45e3132f580c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:11.662504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1299/2616] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:11.672239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5112857-53c4-4b8c-b794-a4d0a42c1025 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:11.672575 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5112857-53c4-4b8c-b794-a4d0a42c1025 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:11.672965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5112857-53c4-4b8c-b794-a4d0a42c1025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:11.704002 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d5112857-53c4-4b8c-b794-a4d0a42c1025 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:11.705307 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d5112857-53c4-4b8c-b794-a4d0a42c1025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:11.714521 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1318/2617] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:11.719546 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e824ba5-c325-496d-8863-45e3132f580c None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:11.734874 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e824ba5-c325-496d-8863-45e3132f580c None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:11.740155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbaaffa-aee5-4c5f-bc6b-541b9263c042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:11.740882 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbaaffa-aee5-4c5f-bc6b-541b9263c042 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:11.741245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbaaffa-aee5-4c5f-bc6b-541b9263c042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:11.741516 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbaaffa-aee5-4c5f-bc6b-541b9263c042 admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:11.742235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbaaffa-aee5-4c5f-bc6b-541b9263c042 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:11.742539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbaaffa-aee5-4c5f-bc6b-541b9263c042 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:11.744247 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbaaffa-aee5-4c5f-bc6b-541b9263c042 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:11.758716 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1300/2618] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:33:11.763684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5112857-53c4-4b8c-b794-a4d0a42c1025 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:11.774010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5112857-53c4-4b8c-b794-a4d0a42c1025 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:11.778729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604b87a-1c0c-4ae8-9557-10dee801598e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:11.779332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604b87a-1c0c-4ae8-9557-10dee801598e admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:11.780542 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604b87a-1c0c-4ae8-9557-10dee801598e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:11.780812 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604b87a-1c0c-4ae8-9557-10dee801598e admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:11.781477 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604b87a-1c0c-4ae8-9557-10dee801598e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:11.783416 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604b87a-1c0c-4ae8-9557-10dee801598e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:11.815320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1319/2619] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:11 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:11.820176 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbaaffa-aee5-4c5f-bc6b-541b9263c042 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:11.830838 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbaaffa-aee5-4c5f-bc6b-541b9263c042 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:11.896728 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678854b-5f6b-4d6e-8e68-bb6466212f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:11.897429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678854b-5f6b-4d6e-8e68-bb6466212f83 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:11.897752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678854b-5f6b-4d6e-8e68-bb6466212f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:11.898007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678854b-5f6b-4d6e-8e68-bb6466212f83 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:11.898708 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678854b-5f6b-4d6e-8e68-bb6466212f83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:11.900821 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678854b-5f6b-4d6e-8e68-bb6466212f83 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:11.925716 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1301/2620] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:11 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:11.930054 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604b87a-1c0c-4ae8-9557-10dee801598e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:11.941195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604b87a-1c0c-4ae8-9557-10dee801598e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:11.945991 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3982a2-f4be-41d6-84cf-c2f9908558fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:11.946592 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3982a2-f4be-41d6-84cf-c2f9908558fd admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:11.946869 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3982a2-f4be-41d6-84cf-c2f9908558fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:11.947194 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3982a2-f4be-41d6-84cf-c2f9908558fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:11.947843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3982a2-f4be-41d6-84cf-c2f9908558fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:11.949483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3982a2-f4be-41d6-84cf-c2f9908558fd admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:11.960554 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1320/2621] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:11.966247 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678854b-5f6b-4d6e-8e68-bb6466212f83 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:11.979203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678854b-5f6b-4d6e-8e68-bb6466212f83 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:11.985974 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ebf19-b17f-4aa0-9049-ed3fb9a6c67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:11.986717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ebf19-b17f-4aa0-9049-ed3fb9a6c67c admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:11.987053 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ebf19-b17f-4aa0-9049-ed3fb9a6c67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:11.987377 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ebf19-b17f-4aa0-9049-ed3fb9a6c67c admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:11.988134 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ebf19-b17f-4aa0-9049-ed3fb9a6c67c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:11.990172 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ebf19-b17f-4aa0-9049-ed3fb9a6c67c admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:11.993042 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:12.004259 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1302/2622] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:12.008456 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3982a2-f4be-41d6-84cf-c2f9908558fd admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:12.019552 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3982a2-f4be-41d6-84cf-c2f9908558fd admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:12.024899 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713f4f0-50c4-4c84-9d42-3bb4bcdea484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:12.025593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713f4f0-50c4-4c84-9d42-3bb4bcdea484 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:12.025876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713f4f0-50c4-4c84-9d42-3bb4bcdea484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:12.026229 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713f4f0-50c4-4c84-9d42-3bb4bcdea484 admin admin] PATH_INFO: `/v3/projects/66f5266455ac46608e5c9c8730a03aa8/users/8c04c05e16634b9299867cad77387404/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:12.043413 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713f4f0-50c4-4c84-9d42-3bb4bcdea484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f5266455ac46608e5c9c8730a03aa8, user_id=8c04c05e16634b9299867cad77387404, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:12.046978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713f4f0-50c4-4c84-9d42-3bb4bcdea484 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:12.063769 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1321/2623] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:12 2022] PUT /identity/v3/projects/66f5266455ac46608e5c9c8730a03aa8/users/8c04c05e16634b9299867cad77387404/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:12.075556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfe8d5-68a0-48e4-939e-62f2973f9dae None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:12.075828 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfe8d5-68a0-48e4-939e-62f2973f9dae None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:12.076090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfe8d5-68a0-48e4-939e-62f2973f9dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:12.109160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-ecbfe8d5-68a0-48e4-939e-62f2973f9dae None None] MFA Rules not processed for user `8c04c05e16634b9299867cad77387404`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:12.110412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ecbfe8d5-68a0-48e4-939e-62f2973f9dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:12.187482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1303/2624] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:12.204048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713f4f0-50c4-4c84-9d42-3bb4bcdea484 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:12.206564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e713f4f0-50c4-4c84-9d42-3bb4bcdea484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:12.289864 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:12.305308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713f4f0-50c4-4c84-9d42-3bb4bcdea484 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:12.311091 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11164777-281d-46ae-add6-20f4b12b1569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:12.312206 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11164777-281d-46ae-add6-20f4b12b1569 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:12.312733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11164777-281d-46ae-add6-20f4b12b1569 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:12.313764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11164777-281d-46ae-add6-20f4b12b1569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:12.331816 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11164777-281d-46ae-add6-20f4b12b1569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:12.334717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11164777-281d-46ae-add6-20f4b12b1569 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:12.357295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1322/2625] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:12.360912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfe8d5-68a0-48e4-939e-62f2973f9dae None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:12.362625 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ecbfe8d5-68a0-48e4-939e-62f2973f9dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:12.384599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11164777-281d-46ae-add6-20f4b12b1569 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:12.475948 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfe8d5-68a0-48e4-939e-62f2973f9dae None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:12.484374 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea24adc-5db8-469d-a61f-b6ab52bb1fae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:12.485381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea24adc-5db8-469d-a61f-b6ab52bb1fae service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:12.485726 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea24adc-5db8-469d-a61f-b6ab52bb1fae service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:12.486127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea24adc-5db8-469d-a61f-b6ab52bb1fae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:12.489202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5ea24adc-5db8-469d-a61f-b6ab52bb1fae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:12.492505 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11164777-281d-46ae-add6-20f4b12b1569 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:12.497619 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c57c5-5271-4d8d-be33-07a7f062bec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:12.498206 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c57c5-5271-4d8d-be33-07a7f062bec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:12.498534 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c57c5-5271-4d8d-be33-07a7f062bec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:12.498918 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c57c5-5271-4d8d-be33-07a7f062bec2 admin admin] PATH_INFO: `/v3/users/8c04c05e16634b9299867cad77387404` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:12.500699 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c57c5-5271-4d8d-be33-07a7f062bec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c04c05e16634b9299867cad77387404)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:12.502738 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c57c5-5271-4d8d-be33-07a7f062bec2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:12.505616 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea24adc-5db8-469d-a61f-b6ab52bb1fae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:12.507393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea24adc-5db8-469d-a61f-b6ab52bb1fae service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:12.525360 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1304/2626] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:12.581500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-959c57c5-5271-4d8d-be33-07a7f062bec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c04c05e16634b9299867cad77387404'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:12.610159 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-959c57c5-5271-4d8d-be33-07a7f062bec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c04c05e16634b9299867cad77387404'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:12.624522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-959c57c5-5271-4d8d-be33-07a7f062bec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c04c05e16634b9299867cad77387404'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:12.625751 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-959c57c5-5271-4d8d-be33-07a7f062bec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c04c05e16634b9299867cad77387404'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:12.650950 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-959c57c5-5271-4d8d-be33-07a7f062bec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c04c05e16634b9299867cad77387404'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:12.660412 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1323/2627] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:12 2022] DELETE /identity/v3/users/8c04c05e16634b9299867cad77387404 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:12.668307 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1516da-f4fd-4069-8ed4-8884b5662476 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:12.668307 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1516da-f4fd-4069-8ed4-8884b5662476 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:12.668732 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1516da-f4fd-4069-8ed4-8884b5662476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:12.700798 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-aa1516da-f4fd-4069-8ed4-8884b5662476 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:12.703344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-aa1516da-f4fd-4069-8ed4-8884b5662476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:12.828122 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1305/2628] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:12.844831 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c57c5-5271-4d8d-be33-07a7f062bec2 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:12.865312 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-959c57c5-5271-4d8d-be33-07a7f062bec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:12.972914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c57c5-5271-4d8d-be33-07a7f062bec2 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:12.977686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0daf3-d271-44d3-a9ab-252b610978df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:12.978546 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0daf3-d271-44d3-a9ab-252b610978df service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:12.978974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0daf3-d271-44d3-a9ab-252b610978df service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:12.979444 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0daf3-d271-44d3-a9ab-252b610978df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:12.992363 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0daf3-d271-44d3-a9ab-252b610978df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:12.994955 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0daf3-d271-44d3-a9ab-252b610978df service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:13.008828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1324/2629] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:13.566010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1516da-f4fd-4069-8ed4-8884b5662476 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:13.567996 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-aa1516da-f4fd-4069-8ed4-8884b5662476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:13.584105 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1516da-f4fd-4069-8ed4-8884b5662476 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:13.589771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069815e1-9efb-4dd4-83bb-fe244316b847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:13.590859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069815e1-9efb-4dd4-83bb-fe244316b847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:13.591351 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069815e1-9efb-4dd4-83bb-fe244316b847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:13.591838 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069815e1-9efb-4dd4-83bb-fe244316b847 admin admin] PATH_INFO: `/v3/projects/66f5266455ac46608e5c9c8730a03aa8` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:13.594940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069815e1-9efb-4dd4-83bb-fe244316b847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66f5266455ac46608e5c9c8730a03aa8)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:13.599032 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069815e1-9efb-4dd4-83bb-fe244316b847 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:13.671859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:13.708007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-069815e1-9efb-4dd4-83bb-fe244316b847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66f5266455ac46608e5c9c8730a03aa8', 'request_id': 'req-069815e1-9efb-4dd4-83bb-fe244316b847'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:13.718100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-069815e1-9efb-4dd4-83bb-fe244316b847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66f5266455ac46608e5c9c8730a03aa8', 'request_id': 'req-069815e1-9efb-4dd4-83bb-fe244316b847'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:13.728052 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-069815e1-9efb-4dd4-83bb-fe244316b847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66f5266455ac46608e5c9c8730a03aa8', 'request_id': 'req-069815e1-9efb-4dd4-83bb-fe244316b847'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:13.753875 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-069815e1-9efb-4dd4-83bb-fe244316b847 admin admin] The token cache is being invalidate because project 66f5266455ac46608e5c9c8730a03aa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:13.754266 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-069815e1-9efb-4dd4-83bb-fe244316b847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:13.764711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-069815e1-9efb-4dd4-83bb-fe244316b847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:13.770124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1306/2630] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:13 2022] DELETE /identity/v3/projects/66f5266455ac46608e5c9c8730a03aa8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:13.772218 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93af4ce-5213-4a49-875c-e62c20eeec98 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:13.772844 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93af4ce-5213-4a49-875c-e62c20eeec98 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:13.772844 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93af4ce-5213-4a49-875c-e62c20eeec98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:13.799269 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e93af4ce-5213-4a49-875c-e62c20eeec98 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:13.800956 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e93af4ce-5213-4a49-875c-e62c20eeec98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:13.810106 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:13.906395 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1325/2631] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:13.918156 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514a4c5-9bae-4cb7-8fe3-5fff8da5c118 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:13.919253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514a4c5-9bae-4cb7-8fe3-5fff8da5c118 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:13.919253 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514a4c5-9bae-4cb7-8fe3-5fff8da5c118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:13.950389 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5514a4c5-9bae-4cb7-8fe3-5fff8da5c118 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:13.953090 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5514a4c5-9bae-4cb7-8fe3-5fff8da5c118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:13.966044 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1307/2632] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:13.972424 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93af4ce-5213-4a49-875c-e62c20eeec98 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:13.992808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93af4ce-5213-4a49-875c-e62c20eeec98 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:13.998165 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb7d56-13ba-4c96-bab0-70a41fb7bd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:13.999815 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb7d56-13ba-4c96-bab0-70a41fb7bd75 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:13.999815 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb7d56-13ba-4c96-bab0-70a41fb7bd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:14.000297 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb7d56-13ba-4c96-bab0-70a41fb7bd75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:14.000885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb7d56-13ba-4c96-bab0-70a41fb7bd75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:14.001223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb7d56-13ba-4c96-bab0-70a41fb7bd75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:14.003042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb7d56-13ba-4c96-bab0-70a41fb7bd75 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:14.019422 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1326/2633] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:33:14.025016 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514a4c5-9bae-4cb7-8fe3-5fff8da5c118 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:14.037699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514a4c5-9bae-4cb7-8fe3-5fff8da5c118 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:14.044052 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6555c5-f3e2-4c50-a704-48fdb362d7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:14.044993 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6555c5-f3e2-4c50-a704-48fdb362d7ab admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:14.046075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6555c5-f3e2-4c50-a704-48fdb362d7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:14.046356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6555c5-f3e2-4c50-a704-48fdb362d7ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:14.047040 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6555c5-f3e2-4c50-a704-48fdb362d7ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:14.049027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6555c5-f3e2-4c50-a704-48fdb362d7ab admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:14.086602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1308/2634] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:14 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:14.090782 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb7d56-13ba-4c96-bab0-70a41fb7bd75 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:14.102253 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb7d56-13ba-4c96-bab0-70a41fb7bd75 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:14.184399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59cfee7-74df-4fb7-a20e-3085a87782ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:14.184954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cfee7-74df-4fb7-a20e-3085a87782ee admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:14.185275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cfee7-74df-4fb7-a20e-3085a87782ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:14.185556 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cfee7-74df-4fb7-a20e-3085a87782ee admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:14.186463 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59cfee7-74df-4fb7-a20e-3085a87782ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:14.189570 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59cfee7-74df-4fb7-a20e-3085a87782ee admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:14.220353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1327/2635] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:14 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:14.225978 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6555c5-f3e2-4c50-a704-48fdb362d7ab admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:14.245493 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6555c5-f3e2-4c50-a704-48fdb362d7ab admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:14.254568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f39134-c2c9-40ae-9056-92d4524fbccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:14.255971 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f39134-c2c9-40ae-9056-92d4524fbccd admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:14.256644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f39134-c2c9-40ae-9056-92d4524fbccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:14.257306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f39134-c2c9-40ae-9056-92d4524fbccd admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:14.258604 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f39134-c2c9-40ae-9056-92d4524fbccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:14.261807 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f39134-c2c9-40ae-9056-92d4524fbccd admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:14.288385 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1309/2636] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:14.290057 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59cfee7-74df-4fb7-a20e-3085a87782ee admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:14.313416 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59cfee7-74df-4fb7-a20e-3085a87782ee admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:14.323170 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccf8bf-02c8-4457-99f2-2451a9a5886a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:14.323942 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccf8bf-02c8-4457-99f2-2451a9a5886a admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:14.324123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccf8bf-02c8-4457-99f2-2451a9a5886a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:14.324393 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccf8bf-02c8-4457-99f2-2451a9a5886a admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:14.325029 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caccf8bf-02c8-4457-99f2-2451a9a5886a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:14.328223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caccf8bf-02c8-4457-99f2-2451a9a5886a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:14.348664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1328/2637] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:14.352311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f39134-c2c9-40ae-9056-92d4524fbccd admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:14.374981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f39134-c2c9-40ae-9056-92d4524fbccd admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:14.383171 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe64da7-c4fb-4e9f-aae6-1ccb497c9cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:14.384329 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe64da7-c4fb-4e9f-aae6-1ccb497c9cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:14.384813 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe64da7-c4fb-4e9f-aae6-1ccb497c9cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:14.385262 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe64da7-c4fb-4e9f-aae6-1ccb497c9cbe admin admin] PATH_INFO: `/v3/projects/f34ba03590a64543b9dc194609b59331/users/3020d6976af544ce9098d6b545cc67a8/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:14.412768 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe64da7-c4fb-4e9f-aae6-1ccb497c9cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f34ba03590a64543b9dc194609b59331, user_id=3020d6976af544ce9098d6b545cc67a8, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:14.417818 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe64da7-c4fb-4e9f-aae6-1ccb497c9cbe admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:14.469177 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1310/2638] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:14 2022] PUT /identity/v3/projects/f34ba03590a64543b9dc194609b59331/users/3020d6976af544ce9098d6b545cc67a8/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:14.476834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e3cde-146c-48ab-a3d9-f7a1a9815cf6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:14.477381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e3cde-146c-48ab-a3d9-f7a1a9815cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:14.477872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e3cde-146c-48ab-a3d9-f7a1a9815cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:14.548244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c62e3cde-146c-48ab-a3d9-f7a1a9815cf6 None None] MFA Rules not processed for user `3020d6976af544ce9098d6b545cc67a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:14.550700 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c62e3cde-146c-48ab-a3d9-f7a1a9815cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:14.694532 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1329/2639] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:14.712380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe64da7-c4fb-4e9f-aae6-1ccb497c9cbe admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:14.715733 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ffe64da7-c4fb-4e9f-aae6-1ccb497c9cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:14.847763 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe64da7-c4fb-4e9f-aae6-1ccb497c9cbe admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:14.853302 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba365d07-7902-47b5-9dd3-f0a7b74c30aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:14.854337 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba365d07-7902-47b5-9dd3-f0a7b74c30aa service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:14.854771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba365d07-7902-47b5-9dd3-f0a7b74c30aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:14.855166 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba365d07-7902-47b5-9dd3-f0a7b74c30aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:14.868661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba365d07-7902-47b5-9dd3-f0a7b74c30aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:14.870762 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba365d07-7902-47b5-9dd3-f0a7b74c30aa service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:14.882854 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1311/2640] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:17.967028 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e3cde-146c-48ab-a3d9-f7a1a9815cf6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:17.970736 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c62e3cde-146c-48ab-a3d9-f7a1a9815cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:17.985606 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:18.102489 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e3cde-146c-48ab-a3d9-f7a1a9815cf6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:18.110343 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca477b0-ec11-444c-9f86-5b5c43b21cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:18.111408 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca477b0-ec11-444c-9f86-5b5c43b21cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:18.111896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca477b0-ec11-444c-9f86-5b5c43b21cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:18.112384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca477b0-ec11-444c-9f86-5b5c43b21cf2 admin admin] PATH_INFO: `/v3/users/c06cd43d533f4621b0a69dbb0ef22dc4` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:18.114612 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca477b0-ec11-444c-9f86-5b5c43b21cf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c06cd43d533f4621b0a69dbb0ef22dc4)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:18.118333 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca477b0-ec11-444c-9f86-5b5c43b21cf2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:18.205746 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bca477b0-ec11-444c-9f86-5b5c43b21cf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c06cd43d533f4621b0a69dbb0ef22dc4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.226958 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bca477b0-ec11-444c-9f86-5b5c43b21cf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c06cd43d533f4621b0a69dbb0ef22dc4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.238655 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bca477b0-ec11-444c-9f86-5b5c43b21cf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c06cd43d533f4621b0a69dbb0ef22dc4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.239363 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bca477b0-ec11-444c-9f86-5b5c43b21cf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c06cd43d533f4621b0a69dbb0ef22dc4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.254210 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bca477b0-ec11-444c-9f86-5b5c43b21cf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c06cd43d533f4621b0a69dbb0ef22dc4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.266480 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1330/2641] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:17 2022] DELETE /identity/v3/users/c06cd43d533f4621b0a69dbb0ef22dc4 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:18.493966 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba365d07-7902-47b5-9dd3-f0a7b74c30aa service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:18.515990 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba365d07-7902-47b5-9dd3-f0a7b74c30aa service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:18.610447 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:18.622160 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02319d4d-9ef7-4084-bbe0-ad80454d33b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:18.624203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02319d4d-9ef7-4084-bbe0-ad80454d33b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:18.624943 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02319d4d-9ef7-4084-bbe0-ad80454d33b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:18.625266 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02319d4d-9ef7-4084-bbe0-ad80454d33b9 admin admin] PATH_INFO: `/v3/projects/8be30316f0b745d783ec96bc983b2cb1` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:18.626865 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02319d4d-9ef7-4084-bbe0-ad80454d33b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8be30316f0b745d783ec96bc983b2cb1)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:18.630392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02319d4d-9ef7-4084-bbe0-ad80454d33b9 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:18.703292 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-02319d4d-9ef7-4084-bbe0-ad80454d33b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8be30316f0b745d783ec96bc983b2cb1', 'request_id': 'req-02319d4d-9ef7-4084-bbe0-ad80454d33b9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.711953 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-02319d4d-9ef7-4084-bbe0-ad80454d33b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8be30316f0b745d783ec96bc983b2cb1', 'request_id': 'req-02319d4d-9ef7-4084-bbe0-ad80454d33b9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.723355 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-02319d4d-9ef7-4084-bbe0-ad80454d33b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8be30316f0b745d783ec96bc983b2cb1', 'request_id': 'req-02319d4d-9ef7-4084-bbe0-ad80454d33b9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.750051 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-02319d4d-9ef7-4084-bbe0-ad80454d33b9 admin admin] The token cache is being invalidate because project 8be30316f0b745d783ec96bc983b2cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:18.750390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-02319d4d-9ef7-4084-bbe0-ad80454d33b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.751131 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-02319d4d-9ef7-4084-bbe0-ad80454d33b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.753205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1312/2642] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:18 2022] DELETE /identity/v3/projects/8be30316f0b745d783ec96bc983b2cb1 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:18.759254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529388f-fab2-4278-8535-63d3b19f8f56 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:18.759569 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529388f-fab2-4278-8535-63d3b19f8f56 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:18.759920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529388f-fab2-4278-8535-63d3b19f8f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:18.789983 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-8529388f-fab2-4278-8535-63d3b19f8f56 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:18.792047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8529388f-fab2-4278-8535-63d3b19f8f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:18.880385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1331/2643] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:18.889601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcc31c-4fce-4bd9-9659-5a73f9fdbad9 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:18.889933 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcc31c-4fce-4bd9-9659-5a73f9fdbad9 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:18.890251 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcc31c-4fce-4bd9-9659-5a73f9fdbad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:18.913126 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-61bcc31c-4fce-4bd9-9659-5a73f9fdbad9 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:18.914357 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-61bcc31c-4fce-4bd9-9659-5a73f9fdbad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:18.923490 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1313/2644] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:18.928755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529388f-fab2-4278-8535-63d3b19f8f56 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:18.942356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529388f-fab2-4278-8535-63d3b19f8f56 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:18.950938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a05d5-5f16-475c-86db-72119574775e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:18.955486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a05d5-5f16-475c-86db-72119574775e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:18.955486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a05d5-5f16-475c-86db-72119574775e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:18.955486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a05d5-5f16-475c-86db-72119574775e admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:18.955486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a05d5-5f16-475c-86db-72119574775e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:18.955486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a05d5-5f16-475c-86db-72119574775e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:18.955486 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a05d5-5f16-475c-86db-72119574775e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:18.970764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1332/2645] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:33:18.976987 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcc31c-4fce-4bd9-9659-5a73f9fdbad9 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:18.987461 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcc31c-4fce-4bd9-9659-5a73f9fdbad9 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:18.992535 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc36328-518d-4470-8142-01b7d24a5dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:18.993190 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc36328-518d-4470-8142-01b7d24a5dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:18.993502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc36328-518d-4470-8142-01b7d24a5dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:18.993875 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc36328-518d-4470-8142-01b7d24a5dd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:18.994898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc36328-518d-4470-8142-01b7d24a5dd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:18.997019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc36328-518d-4470-8142-01b7d24a5dd3 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:19.039554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1314/2646] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:18 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:19.045863 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a05d5-5f16-475c-86db-72119574775e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:19.058072 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a05d5-5f16-475c-86db-72119574775e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:19.070832 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:19.147827 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb944a-8336-476d-b195-8d58a20acc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:19.148463 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb944a-8336-476d-b195-8d58a20acc31 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:19.148765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb944a-8336-476d-b195-8d58a20acc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:19.149146 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb944a-8336-476d-b195-8d58a20acc31 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:19.150253 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb944a-8336-476d-b195-8d58a20acc31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:19.153508 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb944a-8336-476d-b195-8d58a20acc31 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:19.183238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1333/2647] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:19 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:19.188925 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc36328-518d-4470-8142-01b7d24a5dd3 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:19.208583 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc36328-518d-4470-8142-01b7d24a5dd3 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:19.217465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457d11a-fc81-4aed-a73e-add5e3916834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:19.218568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457d11a-fc81-4aed-a73e-add5e3916834 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:19.219107 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457d11a-fc81-4aed-a73e-add5e3916834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:19.219609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457d11a-fc81-4aed-a73e-add5e3916834 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:19.220849 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457d11a-fc81-4aed-a73e-add5e3916834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:19.224101 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457d11a-fc81-4aed-a73e-add5e3916834 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:19.244397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1315/2648] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:19.249461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb944a-8336-476d-b195-8d58a20acc31 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:19.260567 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb944a-8336-476d-b195-8d58a20acc31 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:19.267229 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa782953-64d7-4370-a3cf-955c6039fe74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:19.268065 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa782953-64d7-4370-a3cf-955c6039fe74 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:19.268468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa782953-64d7-4370-a3cf-955c6039fe74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:19.268803 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa782953-64d7-4370-a3cf-955c6039fe74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:19.270175 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa782953-64d7-4370-a3cf-955c6039fe74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:19.272479 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa782953-64d7-4370-a3cf-955c6039fe74 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:19.287042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1334/2649] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:19.292764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457d11a-fc81-4aed-a73e-add5e3916834 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:19.305019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457d11a-fc81-4aed-a73e-add5e3916834 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:19.312169 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdcbcc-d60f-4703-b258-b0ac440d13f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:19.312764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdcbcc-d60f-4703-b258-b0ac440d13f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:19.312995 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdcbcc-d60f-4703-b258-b0ac440d13f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:19.313188 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdcbcc-d60f-4703-b258-b0ac440d13f1 admin admin] PATH_INFO: `/v3/projects/cfac9263b7654b5eb4742ef2638dffc1/users/1bc35c9f116d4de8a2f1bb5cccb81053/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:19.338641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdcbcc-d60f-4703-b258-b0ac440d13f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfac9263b7654b5eb4742ef2638dffc1, user_id=1bc35c9f116d4de8a2f1bb5cccb81053, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:19.342245 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdcbcc-d60f-4703-b258-b0ac440d13f1 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:19.359849 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1316/2650] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:19 2022] PUT /identity/v3/projects/cfac9263b7654b5eb4742ef2638dffc1/users/1bc35c9f116d4de8a2f1bb5cccb81053/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:19.371100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046c663-ae03-44c7-849f-81ae843bddd3 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:19.371392 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046c663-ae03-44c7-849f-81ae843bddd3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:19.371836 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046c663-ae03-44c7-849f-81ae843bddd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:19.428761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7046c663-ae03-44c7-849f-81ae843bddd3 None None] MFA Rules not processed for user `1bc35c9f116d4de8a2f1bb5cccb81053`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:19.431059 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7046c663-ae03-44c7-849f-81ae843bddd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:19.522850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1335/2651] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:19.538294 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdcbcc-d60f-4703-b258-b0ac440d13f1 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:19.540752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ebcdcbcc-d60f-4703-b258-b0ac440d13f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:19.675375 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:19.696926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdcbcc-d60f-4703-b258-b0ac440d13f1 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:19.711717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb336b9-bc1a-40a2-b1d1-97b40cbe9f7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:19.712808 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb336b9-bc1a-40a2-b1d1-97b40cbe9f7a service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:19.713340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb336b9-bc1a-40a2-b1d1-97b40cbe9f7a service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:19.713764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb336b9-bc1a-40a2-b1d1-97b40cbe9f7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:19.726900 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb336b9-bc1a-40a2-b1d1-97b40cbe9f7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:19.729375 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb336b9-bc1a-40a2-b1d1-97b40cbe9f7a service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:19.745406 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1317/2652] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:25.018089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046c663-ae03-44c7-849f-81ae843bddd3 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:25.118286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046c663-ae03-44c7-849f-81ae843bddd3 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:25.125653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3556e-e577-4d2d-b9ce-ea4157de23a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:25.126763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3556e-e577-4d2d-b9ce-ea4157de23a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:25.127246 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3556e-e577-4d2d-b9ce-ea4157de23a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:25.127749 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3556e-e577-4d2d-b9ce-ea4157de23a7 admin admin] PATH_INFO: `/v3/users/1bc35c9f116d4de8a2f1bb5cccb81053` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:25.130039 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3556e-e577-4d2d-b9ce-ea4157de23a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bc35c9f116d4de8a2f1bb5cccb81053)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:25.132382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3556e-e577-4d2d-b9ce-ea4157de23a7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:25.169262 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:25.220667 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f8b3556e-e577-4d2d-b9ce-ea4157de23a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bc35c9f116d4de8a2f1bb5cccb81053'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.247306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f8b3556e-e577-4d2d-b9ce-ea4157de23a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bc35c9f116d4de8a2f1bb5cccb81053'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.260133 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f8b3556e-e577-4d2d-b9ce-ea4157de23a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bc35c9f116d4de8a2f1bb5cccb81053'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.261034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f8b3556e-e577-4d2d-b9ce-ea4157de23a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bc35c9f116d4de8a2f1bb5cccb81053'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.271182 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f8b3556e-e577-4d2d-b9ce-ea4157de23a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bc35c9f116d4de8a2f1bb5cccb81053'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.289896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1336/2653] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:25 2022] DELETE /identity/v3/users/1bc35c9f116d4de8a2f1bb5cccb81053 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:25.297254 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5aad6-9742-4484-8506-9e5e3f6eb74f None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:25.297923 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5aad6-9742-4484-8506-9e5e3f6eb74f None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:25.298230 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5aad6-9742-4484-8506-9e5e3f6eb74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:25.515687 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-28f5aad6-9742-4484-8506-9e5e3f6eb74f None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:25.516871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-28f5aad6-9742-4484-8506-9e5e3f6eb74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:25.570774 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3556e-e577-4d2d-b9ce-ea4157de23a7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:25.572489 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f8b3556e-e577-4d2d-b9ce-ea4157de23a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:25.629535 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1318/2654] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:25.649311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5aad6-9742-4484-8506-9e5e3f6eb74f None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:25.651921 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-28f5aad6-9742-4484-8506-9e5e3f6eb74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:25.709457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3556e-e577-4d2d-b9ce-ea4157de23a7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:25.719148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bab6c-018a-4648-b7ba-dcab25ed2696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:25.720293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bab6c-018a-4648-b7ba-dcab25ed2696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:25.720689 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bab6c-018a-4648-b7ba-dcab25ed2696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:25.721171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bab6c-018a-4648-b7ba-dcab25ed2696 admin admin] PATH_INFO: `/v3/users/3020d6976af544ce9098d6b545cc67a8` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:25.723394 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bab6c-018a-4648-b7ba-dcab25ed2696 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3020d6976af544ce9098d6b545cc67a8)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:25.726915 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bab6c-018a-4648-b7ba-dcab25ed2696 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:25.782279 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5aad6-9742-4484-8506-9e5e3f6eb74f None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:25.787719 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d83a1-1a64-4ebd-a0f0-3e8d4a6e8a84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:25.788684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d83a1-1a64-4ebd-a0f0-3e8d4a6e8a84 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:25.789334 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d83a1-1a64-4ebd-a0f0-3e8d4a6e8a84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:25.789663 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d83a1-1a64-4ebd-a0f0-3e8d4a6e8a84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:25.813698 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d83a1-1a64-4ebd-a0f0-3e8d4a6e8a84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:25.815996 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d83a1-1a64-4ebd-a0f0-3e8d4a6e8a84 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:25.832521 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1319/2655] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:25.836536 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bf7bab6c-018a-4648-b7ba-dcab25ed2696 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3020d6976af544ce9098d6b545cc67a8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.863020 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bf7bab6c-018a-4648-b7ba-dcab25ed2696 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3020d6976af544ce9098d6b545cc67a8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.877210 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bf7bab6c-018a-4648-b7ba-dcab25ed2696 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3020d6976af544ce9098d6b545cc67a8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.878536 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bf7bab6c-018a-4648-b7ba-dcab25ed2696 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3020d6976af544ce9098d6b545cc67a8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.890996 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bf7bab6c-018a-4648-b7ba-dcab25ed2696 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3020d6976af544ce9098d6b545cc67a8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.907754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1337/2656] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:25 2022] DELETE /identity/v3/users/3020d6976af544ce9098d6b545cc67a8 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:25.915162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbda84-5049-4393-9dee-a84e49b12a94 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:25.915484 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbda84-5049-4393-9dee-a84e49b12a94 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:25.916059 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbda84-5049-4393-9dee-a84e49b12a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:25.969975 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-eddbda84-5049-4393-9dee-a84e49b12a94 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:25.974890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-eddbda84-5049-4393-9dee-a84e49b12a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:26.108387 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1320/2657] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:26.122363 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bab6c-018a-4648-b7ba-dcab25ed2696 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:26.124940 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bf7bab6c-018a-4648-b7ba-dcab25ed2696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:26.171566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:26.262905 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bab6c-018a-4648-b7ba-dcab25ed2696 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:26.272967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce417530-1d73-4b91-af79-bbd10a3b964f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:26.272967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce417530-1d73-4b91-af79-bbd10a3b964f service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:26.273850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce417530-1d73-4b91-af79-bbd10a3b964f service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:26.274340 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce417530-1d73-4b91-af79-bbd10a3b964f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:26.300869 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce417530-1d73-4b91-af79-bbd10a3b964f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:26.302792 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce417530-1d73-4b91-af79-bbd10a3b964f service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:26.316528 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1338/2658] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:26.339352 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbda84-5049-4393-9dee-a84e49b12a94 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:26.341655 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-eddbda84-5049-4393-9dee-a84e49b12a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:26.383620 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbda84-5049-4393-9dee-a84e49b12a94 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:26.401036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e16a7b-9e68-429b-8894-f9ea23777d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:26.402932 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e16a7b-9e68-429b-8894-f9ea23777d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:26.403452 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e16a7b-9e68-429b-8894-f9ea23777d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:26.403945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e16a7b-9e68-429b-8894-f9ea23777d87 admin admin] PATH_INFO: `/v3/projects/cfac9263b7654b5eb4742ef2638dffc1` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:26.410281 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e16a7b-9e68-429b-8894-f9ea23777d87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfac9263b7654b5eb4742ef2638dffc1)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:26.414236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e16a7b-9e68-429b-8894-f9ea23777d87 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:26.472048 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:26.536006 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-32e16a7b-9e68-429b-8894-f9ea23777d87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfac9263b7654b5eb4742ef2638dffc1', 'request_id': 'req-32e16a7b-9e68-429b-8894-f9ea23777d87'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:26.556332 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-32e16a7b-9e68-429b-8894-f9ea23777d87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfac9263b7654b5eb4742ef2638dffc1', 'request_id': 'req-32e16a7b-9e68-429b-8894-f9ea23777d87'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:26.569093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-32e16a7b-9e68-429b-8894-f9ea23777d87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfac9263b7654b5eb4742ef2638dffc1', 'request_id': 'req-32e16a7b-9e68-429b-8894-f9ea23777d87'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:26.593833 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-32e16a7b-9e68-429b-8894-f9ea23777d87 admin admin] The token cache is being invalidate because project cfac9263b7654b5eb4742ef2638dffc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:26.594756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-32e16a7b-9e68-429b-8894-f9ea23777d87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:26.598688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-32e16a7b-9e68-429b-8894-f9ea23777d87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:26.605129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1321/2659] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:26 2022] DELETE /identity/v3/projects/cfac9263b7654b5eb4742ef2638dffc1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:26.610729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9079b75-b44f-47b5-87d4-348885d13d83 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:26.611563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9079b75-b44f-47b5-87d4-348885d13d83 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:26.611886 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9079b75-b44f-47b5-87d4-348885d13d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:26.644067 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f9079b75-b44f-47b5-87d4-348885d13d83 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:26.655473 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f9079b75-b44f-47b5-87d4-348885d13d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:26.770126 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1339/2660] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:26.772483 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d579af9-39a8-4248-8656-0196b0bb903a None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:26.772883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d579af9-39a8-4248-8656-0196b0bb903a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:26.773261 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d579af9-39a8-4248-8656-0196b0bb903a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:26.808414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-7d579af9-39a8-4248-8656-0196b0bb903a None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:26.809528 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-7d579af9-39a8-4248-8656-0196b0bb903a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:26.821372 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1322/2661] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:26.829439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9079b75-b44f-47b5-87d4-348885d13d83 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:26.842974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9079b75-b44f-47b5-87d4-348885d13d83 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:26.848463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d579af9-39a8-4248-8656-0196b0bb903a None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:26.849528 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b34eb-c373-4503-9641-78749f881a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:26.850371 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b34eb-c373-4503-9641-78749f881a76 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:26.850846 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b34eb-c373-4503-9641-78749f881a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:26.851007 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-7d579af9-39a8-4248-8656-0196b0bb903a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:26.851168 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b34eb-c373-4503-9641-78749f881a76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:26.851923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b34eb-c373-4503-9641-78749f881a76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:26.852232 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b34eb-c373-4503-9641-78749f881a76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:26.854437 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b34eb-c373-4503-9641-78749f881a76 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:26.876876 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1340/2662] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:33:26.877185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d579af9-39a8-4248-8656-0196b0bb903a None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:26.879184 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b34eb-c373-4503-9641-78749f881a76 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:26.883648 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66f20b-3d77-41f4-be75-d575afdbb2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:26.884884 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66f20b-3d77-41f4-be75-d575afdbb2a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:26.885469 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66f20b-3d77-41f4-be75-d575afdbb2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:26.886038 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66f20b-3d77-41f4-be75-d575afdbb2a9 admin admin] PATH_INFO: `/v3/projects/f34ba03590a64543b9dc194609b59331` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:26.890839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66f20b-3d77-41f4-be75-d575afdbb2a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f34ba03590a64543b9dc194609b59331)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:26.895809 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66f20b-3d77-41f4-be75-d575afdbb2a9 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:26.899663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b34eb-c373-4503-9641-78749f881a76 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:26.907553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33f6f6-8fc7-4021-8c36-5005d3c0f273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:26.908423 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33f6f6-8fc7-4021-8c36-5005d3c0f273 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:26.908888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33f6f6-8fc7-4021-8c36-5005d3c0f273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:26.909370 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33f6f6-8fc7-4021-8c36-5005d3c0f273 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:26.910440 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33f6f6-8fc7-4021-8c36-5005d3c0f273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:26.913717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33f6f6-8fc7-4021-8c36-5005d3c0f273 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:26.972782 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-cf66f20b-3d77-41f4-be75-d575afdbb2a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f34ba03590a64543b9dc194609b59331', 'request_id': 'req-cf66f20b-3d77-41f4-be75-d575afdbb2a9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:26.974010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1341/2663] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:26 2022] POST /identity/v3/projects => generated 363 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:26.978921 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33f6f6-8fc7-4021-8c36-5005d3c0f273 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:26.987772 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-cf66f20b-3d77-41f4-be75-d575afdbb2a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f34ba03590a64543b9dc194609b59331', 'request_id': 'req-cf66f20b-3d77-41f4-be75-d575afdbb2a9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:27.000097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-cf66f20b-3d77-41f4-be75-d575afdbb2a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f34ba03590a64543b9dc194609b59331', 'request_id': 'req-cf66f20b-3d77-41f4-be75-d575afdbb2a9'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:27.026429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-cf66f20b-3d77-41f4-be75-d575afdbb2a9 admin admin] The token cache is being invalidate because project f34ba03590a64543b9dc194609b59331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:27.026940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-cf66f20b-3d77-41f4-be75-d575afdbb2a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:27.028095 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-cf66f20b-3d77-41f4-be75-d575afdbb2a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:27.032066 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1323/2664] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:26 2022] DELETE /identity/v3/projects/f34ba03590a64543b9dc194609b59331 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:27.040900 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27795b6b-d05c-43e5-8207-85ea66bbd8c2 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.040900 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27795b6b-d05c-43e5-8207-85ea66bbd8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.040900 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27795b6b-d05c-43e5-8207-85ea66bbd8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.076959 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-27795b6b-d05c-43e5-8207-85ea66bbd8c2 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:27.079029 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-27795b6b-d05c-43e5-8207-85ea66bbd8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:27.087405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33f6f6-8fc7-4021-8c36-5005d3c0f273 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:27.100015 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1324/2665] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:27.100256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885d5a3-d7be-4593-a147-7b4242f166c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:27.100256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885d5a3-d7be-4593-a147-7b4242f166c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.100256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885d5a3-d7be-4593-a147-7b4242f166c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.100256 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885d5a3-d7be-4593-a147-7b4242f166c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.100638 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4885d5a3-d7be-4593-a147-7b4242f166c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:27.101309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4885d5a3-d7be-4593-a147-7b4242f166c5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.132807 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1342/2666] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:26 2022] POST /identity/v3/users => generated 282 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:27.137852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27795b6b-d05c-43e5-8207-85ea66bbd8c2 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:27.140515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-27795b6b-d05c-43e5-8207-85ea66bbd8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:27.156691 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27795b6b-d05c-43e5-8207-85ea66bbd8c2 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:27.162751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3025183-c4ee-4a62-bf8f-e2020e0715d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:27.163582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3025183-c4ee-4a62-bf8f-e2020e0715d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.163915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3025183-c4ee-4a62-bf8f-e2020e0715d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.164300 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3025183-c4ee-4a62-bf8f-e2020e0715d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.165018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3025183-c4ee-4a62-bf8f-e2020e0715d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:27.166985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3025183-c4ee-4a62-bf8f-e2020e0715d0 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.182652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1325/2667] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:33:27.189518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885d5a3-d7be-4593-a147-7b4242f166c5 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:27.196367 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b2e1e-1b11-4049-b171-751cb37b618a None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.196610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b2e1e-1b11-4049-b171-751cb37b618a None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.196885 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b2e1e-1b11-4049-b171-751cb37b618a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.206105 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885d5a3-d7be-4593-a147-7b4242f166c5 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:27.215927 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ab96b-0176-40b0-a1ac-2506b5ec4d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:27.217593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ab96b-0176-40b0-a1ac-2506b5ec4d97 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.218112 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ab96b-0176-40b0-a1ac-2506b5ec4d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.218621 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ab96b-0176-40b0-a1ac-2506b5ec4d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.219817 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ab96b-0176-40b0-a1ac-2506b5ec4d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:27.223888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ab96b-0176-40b0-a1ac-2506b5ec4d97 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.227436 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-e06b2e1e-1b11-4049-b171-751cb37b618a None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:27.228913 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-e06b2e1e-1b11-4049-b171-751cb37b618a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:27.238134 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1326/2668] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:27.242064 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1343/2669] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:27.242844 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b2e1e-1b11-4049-b171-751cb37b618a None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:27.248118 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ab96b-0176-40b0-a1ac-2506b5ec4d97 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:27.253413 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b2e1e-1b11-4049-b171-751cb37b618a None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:27.258204 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b9360-fd25-4af0-bb7a-351620255a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:27.258834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b9360-fd25-4af0-bb7a-351620255a6b admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.259019 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ab96b-0176-40b0-a1ac-2506b5ec4d97 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:27.259161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b9360-fd25-4af0-bb7a-351620255a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.259472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b9360-fd25-4af0-bb7a-351620255a6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.260140 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b9360-fd25-4af0-bb7a-351620255a6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:27.261184 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b9360-fd25-4af0-bb7a-351620255a6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:27.262807 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b9360-fd25-4af0-bb7a-351620255a6b admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.264441 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163ee1f-e7fc-4b80-9b58-4b81b578976c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:27.265240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163ee1f-e7fc-4b80-9b58-4b81b578976c admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.265599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163ee1f-e7fc-4b80-9b58-4b81b578976c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.265960 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163ee1f-e7fc-4b80-9b58-4b81b578976c admin admin] PATH_INFO: `/v3/projects/a862d251b43f4ebbb37e5145f18be451/users/19bccdfb3b2b4ab5bfe949aabe970327/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.268711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:27.281276 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1327/2670] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:33:27.286898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b9360-fd25-4af0-bb7a-351620255a6b admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:27.291806 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163ee1f-e7fc-4b80-9b58-4b81b578976c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a862d251b43f4ebbb37e5145f18be451, user_id=19bccdfb3b2b4ab5bfe949aabe970327, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:27.294873 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163ee1f-e7fc-4b80-9b58-4b81b578976c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.305362 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b9360-fd25-4af0-bb7a-351620255a6b admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:27.312361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1344/2671] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:27 2022] PUT /identity/v3/projects/a862d251b43f4ebbb37e5145f18be451/users/19bccdfb3b2b4ab5bfe949aabe970327/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:27.324021 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c2531-fc23-48ea-af0f-7007cc6cabf9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.324310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c2531-fc23-48ea-af0f-7007cc6cabf9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.324594 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c2531-fc23-48ea-af0f-7007cc6cabf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.358750 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-028c2531-fc23-48ea-af0f-7007cc6cabf9 None None] MFA Rules not processed for user `19bccdfb3b2b4ab5bfe949aabe970327`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:27.359983 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-028c2531-fc23-48ea-af0f-7007cc6cabf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:27.368845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:27.384926 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17757b93-99ad-45f2-b249-b6e4ef7583ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:27.385659 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17757b93-99ad-45f2-b249-b6e4ef7583ba admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.386014 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17757b93-99ad-45f2-b249-b6e4ef7583ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.386352 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17757b93-99ad-45f2-b249-b6e4ef7583ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.387018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17757b93-99ad-45f2-b249-b6e4ef7583ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:27.389961 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17757b93-99ad-45f2-b249-b6e4ef7583ba admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.427529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1328/2672] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:27 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:27.433010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17757b93-99ad-45f2-b249-b6e4ef7583ba admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:27.463474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1345/2673] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:27.468825 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c2531-fc23-48ea-af0f-7007cc6cabf9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:27.483196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c2531-fc23-48ea-af0f-7007cc6cabf9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:27.537529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17757b93-99ad-45f2-b249-b6e4ef7583ba admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:27.543409 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f13e60-4fe0-45f3-b30e-42db48e53c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:27.544238 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f13e60-4fe0-45f3-b30e-42db48e53c2c admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.545794 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f13e60-4fe0-45f3-b30e-42db48e53c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.546168 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f13e60-4fe0-45f3-b30e-42db48e53c2c admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.546946 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f13e60-4fe0-45f3-b30e-42db48e53c2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:27.549293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f13e60-4fe0-45f3-b30e-42db48e53c2c admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.578912 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1329/2674] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:27 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:27.583414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f13e60-4fe0-45f3-b30e-42db48e53c2c admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:27.599505 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f13e60-4fe0-45f3-b30e-42db48e53c2c admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:27.601850 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9b156-9988-452b-9479-bf9d501ef827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:27.602977 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9b156-9988-452b-9479-bf9d501ef827 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.603765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9b156-9988-452b-9479-bf9d501ef827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.604444 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9b156-9988-452b-9479-bf9d501ef827 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.605681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9b156-9988-452b-9479-bf9d501ef827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:27.606271 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc2e1c-1cee-42df-a0cf-e4c909aeda7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:27.607880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc2e1c-1cee-42df-a0cf-e4c909aeda7d admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.608393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc2e1c-1cee-42df-a0cf-e4c909aeda7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.609250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc2e1c-1cee-42df-a0cf-e4c909aeda7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.610282 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc2e1c-1cee-42df-a0cf-e4c909aeda7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:27.610851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9b156-9988-452b-9479-bf9d501ef827 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.612601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc2e1c-1cee-42df-a0cf-e4c909aeda7d admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.630724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1330/2675] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:33:27.635383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc2e1c-1cee-42df-a0cf-e4c909aeda7d admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:27.655484 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc2e1c-1cee-42df-a0cf-e4c909aeda7d admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:27.663569 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea57b1-c855-4410-b1f7-70eae8cd309d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:27.664600 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea57b1-c855-4410-b1f7-70eae8cd309d admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.665126 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea57b1-c855-4410-b1f7-70eae8cd309d admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.665698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1346/2676] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:27 2022] POST /identity/v3/projects => generated 365 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:27.665892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea57b1-c855-4410-b1f7-70eae8cd309d admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.666934 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea57b1-c855-4410-b1f7-70eae8cd309d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:27.671465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea57b1-c855-4410-b1f7-70eae8cd309d admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.676755 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9b156-9988-452b-9479-bf9d501ef827 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:27.695228 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1331/2677] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:33:27.703041 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea57b1-c855-4410-b1f7-70eae8cd309d admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:27.718306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea57b1-c855-4410-b1f7-70eae8cd309d admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:27.780720 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:27.786638 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9b156-9988-452b-9479-bf9d501ef827 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:27.805314 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b542f7c-a537-4dd2-9cb4-079923b1648d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:27.806096 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b542f7c-a537-4dd2-9cb4-079923b1648d admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.806488 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b542f7c-a537-4dd2-9cb4-079923b1648d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.806871 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b542f7c-a537-4dd2-9cb4-079923b1648d admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.807681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b542f7c-a537-4dd2-9cb4-079923b1648d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:27.810096 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b542f7c-a537-4dd2-9cb4-079923b1648d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.821318 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82e912-b6f8-4223-867b-106eeeca45cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:27.822091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82e912-b6f8-4223-867b-106eeeca45cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.822380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82e912-b6f8-4223-867b-106eeeca45cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.822688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82e912-b6f8-4223-867b-106eeeca45cf admin admin] PATH_INFO: `/v3/projects/c468621fa4aa429aab72886bfcf95ee3/users/992b881e8bb04aa1a30a5d3ff0f1af5b/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.837922 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1347/2678] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:27 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:27.843615 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b542f7c-a537-4dd2-9cb4-079923b1648d admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:27.847938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82e912-b6f8-4223-867b-106eeeca45cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c468621fa4aa429aab72886bfcf95ee3, user_id=992b881e8bb04aa1a30a5d3ff0f1af5b, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:27.852725 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82e912-b6f8-4223-867b-106eeeca45cf admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.868521 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b542f7c-a537-4dd2-9cb4-079923b1648d admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:27.878619 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1332/2679] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:27 2022] PUT /identity/v3/projects/c468621fa4aa429aab72886bfcf95ee3/users/992b881e8bb04aa1a30a5d3ff0f1af5b/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:27.890922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eac3b-f2b7-40d4-be62-fb188a120998 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.891750 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eac3b-f2b7-40d4-be62-fb188a120998 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.891952 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eac3b-f2b7-40d4-be62-fb188a120998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.945503 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5f3eac3b-f2b7-40d4-be62-fb188a120998 None None] MFA Rules not processed for user `992b881e8bb04aa1a30a5d3ff0f1af5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:27.947661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5f3eac3b-f2b7-40d4-be62-fb188a120998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:27.963297 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0ab67-698a-40ea-90bc-554eced44102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:27.964166 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0ab67-698a-40ea-90bc-554eced44102 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.964677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0ab67-698a-40ea-90bc-554eced44102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.965204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0ab67-698a-40ea-90bc-554eced44102 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.966399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0ab67-698a-40ea-90bc-554eced44102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:27.970210 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0ab67-698a-40ea-90bc-554eced44102 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.972586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:27.988744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1348/2680] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:33:27.994121 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0ab67-698a-40ea-90bc-554eced44102 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:28.013231 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0ab67-698a-40ea-90bc-554eced44102 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:28.023204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cf5f7-a526-4943-ad43-4fcdf03bb5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:28.024391 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cf5f7-a526-4943-ad43-4fcdf03bb5b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:28.024859 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cf5f7-a526-4943-ad43-4fcdf03bb5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:28.025397 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cf5f7-a526-4943-ad43-4fcdf03bb5b5 admin admin] PATH_INFO: `/v3/projects/d54f590704424f41923c13214712da9f/users/7cdccbdd53dd48fa8052bda82b832d43/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:28.052915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1333/2681] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:28.059026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3cf5f7-a526-4943-ad43-4fcdf03bb5b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d54f590704424f41923c13214712da9f, user_id=7cdccbdd53dd48fa8052bda82b832d43, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:28.065493 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3eac3b-f2b7-40d4-be62-fb188a120998 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:28.066485 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3cf5f7-a526-4943-ad43-4fcdf03bb5b5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:28.066992 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5f3eac3b-f2b7-40d4-be62-fb188a120998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:28.085599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1349/2682] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:27 2022] PUT /identity/v3/projects/d54f590704424f41923c13214712da9f/users/7cdccbdd53dd48fa8052bda82b832d43/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:28.092538 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cf5f7-a526-4943-ad43-4fcdf03bb5b5 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:28.176566 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3eac3b-f2b7-40d4-be62-fb188a120998 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:28.193795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cf5f7-a526-4943-ad43-4fcdf03bb5b5 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:28.202796 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68908f36-4ac8-4fed-b63b-3cd8139e269f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:28.203881 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68908f36-4ac8-4fed-b63b-3cd8139e269f admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:28.204384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68908f36-4ac8-4fed-b63b-3cd8139e269f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:28.205468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68908f36-4ac8-4fed-b63b-3cd8139e269f admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:28.206417 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68908f36-4ac8-4fed-b63b-3cd8139e269f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:28.209601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68908f36-4ac8-4fed-b63b-3cd8139e269f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:28.229145 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1350/2683] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:33:28.231638 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68908f36-4ac8-4fed-b63b-3cd8139e269f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:28.246865 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68908f36-4ac8-4fed-b63b-3cd8139e269f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:28.253991 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad206b1-35ef-4c6c-9041-16d8da06f549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:28.254746 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59080022-01f3-4697-abba-0efbe0b7d6e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:28.254990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad206b1-35ef-4c6c-9041-16d8da06f549 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:28.255323 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad206b1-35ef-4c6c-9041-16d8da06f549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:28.255719 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59080022-01f3-4697-abba-0efbe0b7d6e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:28.255951 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad206b1-35ef-4c6c-9041-16d8da06f549 admin admin] PATH_INFO: `/v3/projects/d54f590704424f41923c13214712da9f/users/7cdccbdd53dd48fa8052bda82b832d43/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:28.256111 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59080022-01f3-4697-abba-0efbe0b7d6e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:28.256293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59080022-01f3-4697-abba-0efbe0b7d6e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:28.258933 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad206b1-35ef-4c6c-9041-16d8da06f549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d54f590704424f41923c13214712da9f, user_id=7cdccbdd53dd48fa8052bda82b832d43, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:28.265259 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad206b1-35ef-4c6c-9041-16d8da06f549 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:28.285510 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1351/2684] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:28 2022] PUT /identity/v3/projects/d54f590704424f41923c13214712da9f/users/7cdccbdd53dd48fa8052bda82b832d43/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:28.300631 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d879a-4f95-4d6d-b230-205fa7923b03 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:28.301118 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d879a-4f95-4d6d-b230-205fa7923b03 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:28.301376 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d879a-4f95-4d6d-b230-205fa7923b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:28.350004 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fe6d879a-4f95-4d6d-b230-205fa7923b03 None None] MFA Rules not processed for user `7cdccbdd53dd48fa8052bda82b832d43`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:28.352423 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fe6d879a-4f95-4d6d-b230-205fa7923b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:28.359845 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59080022-01f3-4697-abba-0efbe0b7d6e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:28.361602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59080022-01f3-4697-abba-0efbe0b7d6e2 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:28.375067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1334/2685] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:28.413158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:28.469776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1352/2686] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:28.485231 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59080022-01f3-4697-abba-0efbe0b7d6e2 service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:28.486935 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-59080022-01f3-4697-abba-0efbe0b7d6e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:28.506264 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59080022-01f3-4697-abba-0efbe0b7d6e2 service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:28.543711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:28.578955 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5133a-3ff5-4187-ba48-5bd5f6feb02b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:28.579775 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5133a-3ff5-4187-ba48-5bd5f6feb02b service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:28.580116 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5133a-3ff5-4187-ba48-5bd5f6feb02b service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:28.580425 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5133a-3ff5-4187-ba48-5bd5f6feb02b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:28.597319 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5133a-3ff5-4187-ba48-5bd5f6feb02b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:28.600162 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5133a-3ff5-4187-ba48-5bd5f6feb02b service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:28.615215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1335/2687] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:28 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:28.751485 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d879a-4f95-4d6d-b230-205fa7923b03 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:28.775599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d879a-4f95-4d6d-b230-205fa7923b03 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:28.785756 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf44849-dc19-4776-8493-189f47be2ddd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:28.787014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf44849-dc19-4776-8493-189f47be2ddd service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:28.787622 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf44849-dc19-4776-8493-189f47be2ddd service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:28.788191 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf44849-dc19-4776-8493-189f47be2ddd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:28.871703 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf44849-dc19-4776-8493-189f47be2ddd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:28.874026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf44849-dc19-4776-8493-189f47be2ddd service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:28.885664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1353/2688] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:38.608753 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=14 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:38.881211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=43 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:43.202497 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5133a-3ff5-4187-ba48-5bd5f6feb02b service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:43.204252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-45d5133a-3ff5-4187-ba48-5bd5f6feb02b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:43.322161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5133a-3ff5-4187-ba48-5bd5f6feb02b service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:43.330553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f021c-b103-4d40-b3c0-3fb6505981ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:43.331710 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f021c-b103-4d40-b3c0-3fb6505981ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:43.332396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f021c-b103-4d40-b3c0-3fb6505981ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:43.332693 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f021c-b103-4d40-b3c0-3fb6505981ee admin admin] PATH_INFO: `/v3/users/02f9f109650f484db0357ec6a2154a41` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:43.334102 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f021c-b103-4d40-b3c0-3fb6505981ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=02f9f109650f484db0357ec6a2154a41)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:43.336097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f021c-b103-4d40-b3c0-3fb6505981ee admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:43.405830 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8a0f021c-b103-4d40-b3c0-3fb6505981ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02f9f109650f484db0357ec6a2154a41'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:43.406612 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8a0f021c-b103-4d40-b3c0-3fb6505981ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02f9f109650f484db0357ec6a2154a41'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:43.416291 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8a0f021c-b103-4d40-b3c0-3fb6505981ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02f9f109650f484db0357ec6a2154a41'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:43.436108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8a0f021c-b103-4d40-b3c0-3fb6505981ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02f9f109650f484db0357ec6a2154a41'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:43.447875 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8a0f021c-b103-4d40-b3c0-3fb6505981ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02f9f109650f484db0357ec6a2154a41'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:43.456334 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1336/2689] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:43 2022] DELETE /identity/v3/users/02f9f109650f484db0357ec6a2154a41 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:43.463899 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b0aba-a4d8-47a5-a95e-1e21e1ed69e6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:43.464977 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b0aba-a4d8-47a5-a95e-1e21e1ed69e6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:43.464977 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b0aba-a4d8-47a5-a95e-1e21e1ed69e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:43.494390 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-473b0aba-a4d8-47a5-a95e-1e21e1ed69e6 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:43.495723 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-473b0aba-a4d8-47a5-a95e-1e21e1ed69e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:43.601016 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1354/2690] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:43.615681 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f021c-b103-4d40-b3c0-3fb6505981ee admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:43.617604 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8a0f021c-b103-4d40-b3c0-3fb6505981ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:43.708965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f021c-b103-4d40-b3c0-3fb6505981ee admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:43.714085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442c65c-3117-4882-a521-9fd465c4a39e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:43.714798 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442c65c-3117-4882-a521-9fd465c4a39e service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:43.715473 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442c65c-3117-4882-a521-9fd465c4a39e service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:43.715724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442c65c-3117-4882-a521-9fd465c4a39e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:43.730964 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442c65c-3117-4882-a521-9fd465c4a39e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:43.733864 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442c65c-3117-4882-a521-9fd465c4a39e service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:43.750588 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1337/2691] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:44.143011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b0aba-a4d8-47a5-a95e-1e21e1ed69e6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:44.145163 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-473b0aba-a4d8-47a5-a95e-1e21e1ed69e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:44.163082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b0aba-a4d8-47a5-a95e-1e21e1ed69e6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:44.170004 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ab0d9-f9cf-4a1e-97be-d85fc8688592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:44.170547 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ab0d9-f9cf-4a1e-97be-d85fc8688592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:44.170715 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ab0d9-f9cf-4a1e-97be-d85fc8688592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:44.171206 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ab0d9-f9cf-4a1e-97be-d85fc8688592 admin admin] PATH_INFO: `/v3/users/19bccdfb3b2b4ab5bfe949aabe970327` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:44.172361 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ab0d9-f9cf-4a1e-97be-d85fc8688592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19bccdfb3b2b4ab5bfe949aabe970327)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:44.174798 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ab0d9-f9cf-4a1e-97be-d85fc8688592 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:44.182379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442c65c-3117-4882-a521-9fd465c4a39e service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:44.184582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a442c65c-3117-4882-a521-9fd465c4a39e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:44.213786 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442c65c-3117-4882-a521-9fd465c4a39e service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:44.221116 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7236b9-0cc3-4509-bec9-c13ec2f97052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:44.221794 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7236b9-0cc3-4509-bec9-c13ec2f97052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:44.224539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7236b9-0cc3-4509-bec9-c13ec2f97052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:44.224539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7236b9-0cc3-4509-bec9-c13ec2f97052 admin admin] PATH_INFO: `/v3/projects/faea4df22653419d8b6037254dc070f0` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:44.224539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7236b9-0cc3-4509-bec9-c13ec2f97052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faea4df22653419d8b6037254dc070f0)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:44.226527 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7236b9-0cc3-4509-bec9-c13ec2f97052 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:44.260286 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:44.269633 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-325ab0d9-f9cf-4a1e-97be-d85fc8688592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19bccdfb3b2b4ab5bfe949aabe970327'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:44.288238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-325ab0d9-f9cf-4a1e-97be-d85fc8688592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19bccdfb3b2b4ab5bfe949aabe970327'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:44.302914 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-325ab0d9-f9cf-4a1e-97be-d85fc8688592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19bccdfb3b2b4ab5bfe949aabe970327'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:44.304249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-325ab0d9-f9cf-4a1e-97be-d85fc8688592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19bccdfb3b2b4ab5bfe949aabe970327'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:44.312641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-2f7236b9-0cc3-4509-bec9-c13ec2f97052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faea4df22653419d8b6037254dc070f0', 'request_id': 'req-2f7236b9-0cc3-4509-bec9-c13ec2f97052'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:44.322499 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-325ab0d9-f9cf-4a1e-97be-d85fc8688592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19bccdfb3b2b4ab5bfe949aabe970327'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:44.323533 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-2f7236b9-0cc3-4509-bec9-c13ec2f97052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faea4df22653419d8b6037254dc070f0', 'request_id': 'req-2f7236b9-0cc3-4509-bec9-c13ec2f97052'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:44.334017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1355/2692] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:44 2022] DELETE /identity/v3/users/19bccdfb3b2b4ab5bfe949aabe970327 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:44.338642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-2f7236b9-0cc3-4509-bec9-c13ec2f97052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faea4df22653419d8b6037254dc070f0', 'request_id': 'req-2f7236b9-0cc3-4509-bec9-c13ec2f97052'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:44.342971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9e8a3-331a-4045-bad6-10efb12086d8 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:44.343569 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9e8a3-331a-4045-bad6-10efb12086d8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:44.344115 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9e8a3-331a-4045-bad6-10efb12086d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:44.361185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-2f7236b9-0cc3-4509-bec9-c13ec2f97052 admin admin] The token cache is being invalidate because project faea4df22653419d8b6037254dc070f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:44.361737 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-2f7236b9-0cc3-4509-bec9-c13ec2f97052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:44.362929 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-2f7236b9-0cc3-4509-bec9-c13ec2f97052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:44.365612 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1338/2693] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:44 2022] DELETE /identity/v3/projects/faea4df22653419d8b6037254dc070f0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:44.373386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b080405-0cc0-411c-8a74-c25c15354fff None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:44.373907 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b080405-0cc0-411c-8a74-c25c15354fff None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:44.374061 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b080405-0cc0-411c-8a74-c25c15354fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:44.380040 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fdd9e8a3-331a-4045-bad6-10efb12086d8 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:44.382455 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fdd9e8a3-331a-4045-bad6-10efb12086d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:44.409081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-7b080405-0cc0-411c-8a74-c25c15354fff None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:44.411269 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-7b080405-0cc0-411c-8a74-c25c15354fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:44.487527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1356/2694] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:44.502953 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9e8a3-331a-4045-bad6-10efb12086d8 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:44.504704 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fdd9e8a3-331a-4045-bad6-10efb12086d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:44.516200 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1339/2695] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:44.526657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c014f9a-669d-4e5d-a432-3cc7fc84eadc None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:44.526936 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c014f9a-669d-4e5d-a432-3cc7fc84eadc None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:44.527210 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c014f9a-669d-4e5d-a432-3cc7fc84eadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:44.527832 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:44.555134 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-8c014f9a-669d-4e5d-a432-3cc7fc84eadc None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:44.556664 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8c014f9a-669d-4e5d-a432-3cc7fc84eadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:44.567823 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1340/2696] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:44.572618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c014f9a-669d-4e5d-a432-3cc7fc84eadc None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:44.586505 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c014f9a-669d-4e5d-a432-3cc7fc84eadc None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:44.592084 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8ac8f-62ad-4a87-b6ec-f4eef2b8eb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:44.592706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8ac8f-62ad-4a87-b6ec-f4eef2b8eb52 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:44.592990 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8ac8f-62ad-4a87-b6ec-f4eef2b8eb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:44.593293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8ac8f-62ad-4a87-b6ec-f4eef2b8eb52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:44.594025 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb8ac8f-62ad-4a87-b6ec-f4eef2b8eb52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:44.594323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb8ac8f-62ad-4a87-b6ec-f4eef2b8eb52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:44.595423 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9e8a3-331a-4045-bad6-10efb12086d8 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:44.595941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb8ac8f-62ad-4a87-b6ec-f4eef2b8eb52 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:44.600851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee7e22-f79f-4a66-accd-68bd6a286cd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:44.601586 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee7e22-f79f-4a66-accd-68bd6a286cd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:44.601888 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee7e22-f79f-4a66-accd-68bd6a286cd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:44.602167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee7e22-f79f-4a66-accd-68bd6a286cd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:44.615469 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee7e22-f79f-4a66-accd-68bd6a286cd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:44.617295 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1341/2697] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:33:44.617528 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee7e22-f79f-4a66-accd-68bd6a286cd2 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:44.621878 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8ac8f-62ad-4a87-b6ec-f4eef2b8eb52 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:44.631203 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1357/2698] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:44.634222 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8ac8f-62ad-4a87-b6ec-f4eef2b8eb52 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:44.641272 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9701556-7976-463e-90d1-e2bb508b65d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:44.642010 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9701556-7976-463e-90d1-e2bb508b65d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:44.642344 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9701556-7976-463e-90d1-e2bb508b65d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:44.642671 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9701556-7976-463e-90d1-e2bb508b65d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:44.643387 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9701556-7976-463e-90d1-e2bb508b65d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:44.645590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9701556-7976-463e-90d1-e2bb508b65d6 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:44.697520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1342/2699] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:44 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:44.706893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee7e22-f79f-4a66-accd-68bd6a286cd2 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:44.728017 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee7e22-f79f-4a66-accd-68bd6a286cd2 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:44.860407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44352f62-3374-47e4-aea8-b4280145234b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:44.861682 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44352f62-3374-47e4-aea8-b4280145234b admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:44.862116 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44352f62-3374-47e4-aea8-b4280145234b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:44.862537 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44352f62-3374-47e4-aea8-b4280145234b admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:44.863870 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44352f62-3374-47e4-aea8-b4280145234b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:44.866096 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44352f62-3374-47e4-aea8-b4280145234b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:44.901216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1358/2700] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:44 2022] POST /identity/v3/users => generated 288 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:44.906364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9701556-7976-463e-90d1-e2bb508b65d6 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:44.920150 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9701556-7976-463e-90d1-e2bb508b65d6 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:44.926638 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff6562-5e0a-4593-95c5-bcc1f1a43093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:44.927432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff6562-5e0a-4593-95c5-bcc1f1a43093 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:44.927752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff6562-5e0a-4593-95c5-bcc1f1a43093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:44.928115 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff6562-5e0a-4593-95c5-bcc1f1a43093 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:44.929216 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff6562-5e0a-4593-95c5-bcc1f1a43093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:44.931853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff6562-5e0a-4593-95c5-bcc1f1a43093 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:44.949511 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1343/2701] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:44.955623 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44352f62-3374-47e4-aea8-b4280145234b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:44.968058 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44352f62-3374-47e4-aea8-b4280145234b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:44.985831 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb934f5-37dc-415a-84a7-9de46373c0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:44.986375 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb934f5-37dc-415a-84a7-9de46373c0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:44.986663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb934f5-37dc-415a-84a7-9de46373c0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:44.986941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb934f5-37dc-415a-84a7-9de46373c0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:44.987868 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb934f5-37dc-415a-84a7-9de46373c0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:44.990147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb934f5-37dc-415a-84a7-9de46373c0d7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:45.014199 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1359/2702] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:45.024337 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff6562-5e0a-4593-95c5-bcc1f1a43093 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:45.024563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb934f5-37dc-415a-84a7-9de46373c0d7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:45.027599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6fb934f5-37dc-415a-84a7-9de46373c0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:45.066715 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff6562-5e0a-4593-95c5-bcc1f1a43093 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:45.074219 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb934f5-37dc-415a-84a7-9de46373c0d7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:45.083381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f019351-06de-4d09-b6b2-6d695eb8a24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:45.086733 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50550e-ade4-40eb-9c79-37050a04db83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:45.088100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50550e-ade4-40eb-9c79-37050a04db83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:45.090204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50550e-ade4-40eb-9c79-37050a04db83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:45.092249 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50550e-ade4-40eb-9c79-37050a04db83 admin admin] PATH_INFO: `/v3/projects/a862d251b43f4ebbb37e5145f18be451` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:45.093665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f019351-06de-4d09-b6b2-6d695eb8a24f admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:45.094349 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f019351-06de-4d09-b6b2-6d695eb8a24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:45.095664 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f019351-06de-4d09-b6b2-6d695eb8a24f admin admin] PATH_INFO: `/v3/projects/a07e102f326541e2a89cfe541aa7a022/users/086e60f669894290be19bd9b4b40d1d5/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:45.096920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50550e-ade4-40eb-9c79-37050a04db83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a862d251b43f4ebbb37e5145f18be451)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:45.100648 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50550e-ade4-40eb-9c79-37050a04db83 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:45.135035 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f019351-06de-4d09-b6b2-6d695eb8a24f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07e102f326541e2a89cfe541aa7a022, user_id=086e60f669894290be19bd9b4b40d1d5, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:45.140407 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f019351-06de-4d09-b6b2-6d695eb8a24f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:45.162101 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1344/2703] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:45 2022] PUT /identity/v3/projects/a07e102f326541e2a89cfe541aa7a022/users/086e60f669894290be19bd9b4b40d1d5/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:45.177107 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7a50550e-ade4-40eb-9c79-37050a04db83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a862d251b43f4ebbb37e5145f18be451', 'request_id': 'req-7a50550e-ade4-40eb-9c79-37050a04db83'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.177609 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d65531-3c33-427c-9aad-d9756dab8957 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:45.178071 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d65531-3c33-427c-9aad-d9756dab8957 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:45.178511 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d65531-3c33-427c-9aad-d9756dab8957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:45.188454 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7a50550e-ade4-40eb-9c79-37050a04db83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a862d251b43f4ebbb37e5145f18be451', 'request_id': 'req-7a50550e-ade4-40eb-9c79-37050a04db83'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.201388 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7a50550e-ade4-40eb-9c79-37050a04db83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a862d251b43f4ebbb37e5145f18be451', 'request_id': 'req-7a50550e-ade4-40eb-9c79-37050a04db83'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.220113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-62d65531-3c33-427c-9aad-d9756dab8957 None None] MFA Rules not processed for user `086e60f669894290be19bd9b4b40d1d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:45.221520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-62d65531-3c33-427c-9aad-d9756dab8957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:45.222837 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7a50550e-ade4-40eb-9c79-37050a04db83 admin admin] The token cache is being invalidate because project a862d251b43f4ebbb37e5145f18be451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:45.223366 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7a50550e-ade4-40eb-9c79-37050a04db83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.224498 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7a50550e-ade4-40eb-9c79-37050a04db83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.227761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1360/2704] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:45 2022] DELETE /identity/v3/projects/a862d251b43f4ebbb37e5145f18be451 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:45.234383 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50550e-ade4-40eb-9c79-37050a04db83 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:45.236410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7a50550e-ade4-40eb-9c79-37050a04db83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:45.251429 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:45.266478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:45.334185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1345/2705] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:45.339384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50550e-ade4-40eb-9c79-37050a04db83 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:45.349787 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee077b59-99f9-41e3-ac10-18afd5948ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:45.350955 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee077b59-99f9-41e3-ac10-18afd5948ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:45.351461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee077b59-99f9-41e3-ac10-18afd5948ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:45.351951 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee077b59-99f9-41e3-ac10-18afd5948ba0 admin admin] PATH_INFO: `/v3/users/7cdccbdd53dd48fa8052bda82b832d43` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:45.352998 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d65531-3c33-427c-9aad-d9756dab8957 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:45.354189 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee077b59-99f9-41e3-ac10-18afd5948ba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cdccbdd53dd48fa8052bda82b832d43)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:45.354649 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-62d65531-3c33-427c-9aad-d9756dab8957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:45.359690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee077b59-99f9-41e3-ac10-18afd5948ba0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:45.447313 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-ee077b59-99f9-41e3-ac10-18afd5948ba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cdccbdd53dd48fa8052bda82b832d43'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.448462 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d65531-3c33-427c-9aad-d9756dab8957 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:45.454697 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156befb8-471a-4bd5-880f-73648080ee59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:45.455326 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156befb8-471a-4bd5-880f-73648080ee59 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:45.455604 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156befb8-471a-4bd5-880f-73648080ee59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:45.455879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156befb8-471a-4bd5-880f-73648080ee59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:45.465825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156befb8-471a-4bd5-880f-73648080ee59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:45.467689 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156befb8-471a-4bd5-880f-73648080ee59 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:45.470977 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-ee077b59-99f9-41e3-ac10-18afd5948ba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cdccbdd53dd48fa8052bda82b832d43'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.479532 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1346/2706] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:45.483514 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-ee077b59-99f9-41e3-ac10-18afd5948ba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cdccbdd53dd48fa8052bda82b832d43'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.484329 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-ee077b59-99f9-41e3-ac10-18afd5948ba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cdccbdd53dd48fa8052bda82b832d43'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.496439 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-ee077b59-99f9-41e3-ac10-18afd5948ba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cdccbdd53dd48fa8052bda82b832d43'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.504560 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1361/2707] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:45 2022] DELETE /identity/v3/users/7cdccbdd53dd48fa8052bda82b832d43 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:45.971275 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156befb8-471a-4bd5-880f-73648080ee59 service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:45.996516 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156befb8-471a-4bd5-880f-73648080ee59 service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:46.123047 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4f1ce-0d4d-434b-944d-34911d5e4a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:46.124421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4f1ce-0d4d-434b-944d-34911d5e4a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:46.124931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4f1ce-0d4d-434b-944d-34911d5e4a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:46.125476 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4f1ce-0d4d-434b-944d-34911d5e4a51 admin admin] PATH_INFO: `/v3/projects/d54f590704424f41923c13214712da9f` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:46.127789 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4f1ce-0d4d-434b-944d-34911d5e4a51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d54f590704424f41923c13214712da9f)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:46.131847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4f1ce-0d4d-434b-944d-34911d5e4a51 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:46.219263 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-70b4f1ce-0d4d-434b-944d-34911d5e4a51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd54f590704424f41923c13214712da9f', 'request_id': 'req-70b4f1ce-0d4d-434b-944d-34911d5e4a51'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:46.232140 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-70b4f1ce-0d4d-434b-944d-34911d5e4a51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd54f590704424f41923c13214712da9f', 'request_id': 'req-70b4f1ce-0d4d-434b-944d-34911d5e4a51'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:46.242913 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-70b4f1ce-0d4d-434b-944d-34911d5e4a51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd54f590704424f41923c13214712da9f', 'request_id': 'req-70b4f1ce-0d4d-434b-944d-34911d5e4a51'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:46.245706 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:46.262876 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-70b4f1ce-0d4d-434b-944d-34911d5e4a51 admin admin] The token cache is being invalidate because project d54f590704424f41923c13214712da9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:46.263754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-70b4f1ce-0d4d-434b-944d-34911d5e4a51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:46.264644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-70b4f1ce-0d4d-434b-944d-34911d5e4a51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:46.266389 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1347/2708] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:45 2022] DELETE /identity/v3/projects/d54f590704424f41923c13214712da9f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:46.275871 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873d976-816e-49dd-b740-ffe24c381741 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:46.276151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873d976-816e-49dd-b740-ffe24c381741 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:46.276423 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873d976-816e-49dd-b740-ffe24c381741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:46.303080 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0873d976-816e-49dd-b740-ffe24c381741 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:46.304921 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0873d976-816e-49dd-b740-ffe24c381741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:46.395688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1362/2709] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:46.406086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40601c-2e09-4bfc-8e0e-186b4001b633 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:46.406604 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40601c-2e09-4bfc-8e0e-186b4001b633 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:46.407085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40601c-2e09-4bfc-8e0e-186b4001b633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:46.432860 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-ff40601c-2e09-4bfc-8e0e-186b4001b633 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:46.434132 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-ff40601c-2e09-4bfc-8e0e-186b4001b633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:46.447642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1348/2710] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:46.452887 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873d976-816e-49dd-b740-ffe24c381741 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:46.468081 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873d976-816e-49dd-b740-ffe24c381741 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:46.474039 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee641e-a17b-45d3-90dc-02b66dcdf696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:46.474808 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee641e-a17b-45d3-90dc-02b66dcdf696 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:46.475267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee641e-a17b-45d3-90dc-02b66dcdf696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:46.475770 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee641e-a17b-45d3-90dc-02b66dcdf696 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:46.476737 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee641e-a17b-45d3-90dc-02b66dcdf696 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:46.477048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee641e-a17b-45d3-90dc-02b66dcdf696 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:46.480027 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee641e-a17b-45d3-90dc-02b66dcdf696 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:46.500026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1363/2711] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:33:46.506985 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40601c-2e09-4bfc-8e0e-186b4001b633 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:46.523924 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40601c-2e09-4bfc-8e0e-186b4001b633 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:46.533139 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d5d72-3f3f-4b3b-ac4b-7ec78e6c8532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:46.533998 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d5d72-3f3f-4b3b-ac4b-7ec78e6c8532 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:46.534496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d5d72-3f3f-4b3b-ac4b-7ec78e6c8532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:46.534983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d5d72-3f3f-4b3b-ac4b-7ec78e6c8532 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:46.536165 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d5d72-3f3f-4b3b-ac4b-7ec78e6c8532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:46.540144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d5d72-3f3f-4b3b-ac4b-7ec78e6c8532 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:46.583474 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1349/2712] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:46 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:46.590398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee641e-a17b-45d3-90dc-02b66dcdf696 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:46.608212 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee641e-a17b-45d3-90dc-02b66dcdf696 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:46.649010 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:46.712074 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8866d-5763-4452-9d22-ff68623522fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:46.712819 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8866d-5763-4452-9d22-ff68623522fb admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:46.713179 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8866d-5763-4452-9d22-ff68623522fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:46.713526 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8866d-5763-4452-9d22-ff68623522fb admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:46.714281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8866d-5763-4452-9d22-ff68623522fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:46.717222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8866d-5763-4452-9d22-ff68623522fb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:46.756239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1364/2713] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:46 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:46.761049 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d5d72-3f3f-4b3b-ac4b-7ec78e6c8532 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:46.776131 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d5d72-3f3f-4b3b-ac4b-7ec78e6c8532 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:46.783977 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87753dd-3f62-4e5f-b685-b83fec5b9bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:46.785075 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87753dd-3f62-4e5f-b685-b83fec5b9bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:46.785594 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87753dd-3f62-4e5f-b685-b83fec5b9bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:46.786098 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87753dd-3f62-4e5f-b685-b83fec5b9bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:46.787277 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87753dd-3f62-4e5f-b685-b83fec5b9bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:46.788972 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87753dd-3f62-4e5f-b685-b83fec5b9bc2 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:46.801650 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1350/2714] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:46.805754 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8866d-5763-4452-9d22-ff68623522fb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:46.818250 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8866d-5763-4452-9d22-ff68623522fb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:46.823748 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b78898a-68e2-42f7-b6df-91012e72c51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:46.824366 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78898a-68e2-42f7-b6df-91012e72c51c admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:46.824649 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78898a-68e2-42f7-b6df-91012e72c51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:46.824925 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78898a-68e2-42f7-b6df-91012e72c51c admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:46.826668 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78898a-68e2-42f7-b6df-91012e72c51c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:46.828407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78898a-68e2-42f7-b6df-91012e72c51c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:46.845496 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1365/2715] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:46.850270 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87753dd-3f62-4e5f-b685-b83fec5b9bc2 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:46.861421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87753dd-3f62-4e5f-b685-b83fec5b9bc2 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:46.866149 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e9941-b6f2-4a79-a4ba-8f1767fe6244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:46.866809 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e9941-b6f2-4a79-a4ba-8f1767fe6244 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:46.867088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e9941-b6f2-4a79-a4ba-8f1767fe6244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:46.867361 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e9941-b6f2-4a79-a4ba-8f1767fe6244 admin admin] PATH_INFO: `/v3/projects/c13d31d53cd747308f43e27a32f279a7/users/876874d9ff5b4a9b8248f4ef6366430e/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:46.883871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e9941-b6f2-4a79-a4ba-8f1767fe6244 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c13d31d53cd747308f43e27a32f279a7, user_id=876874d9ff5b4a9b8248f4ef6366430e, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:46.886567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e9941-b6f2-4a79-a4ba-8f1767fe6244 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:46.906249 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1351/2716] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:46 2022] PUT /identity/v3/projects/c13d31d53cd747308f43e27a32f279a7/users/876874d9ff5b4a9b8248f4ef6366430e/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:46.916767 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbaea99-7129-4437-88c2-3c9b97454324 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:46.917047 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbaea99-7129-4437-88c2-3c9b97454324 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:46.917351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbaea99-7129-4437-88c2-3c9b97454324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:46.952041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-bdbaea99-7129-4437-88c2-3c9b97454324 None None] MFA Rules not processed for user `876874d9ff5b4a9b8248f4ef6366430e`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:46.953432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bdbaea99-7129-4437-88c2-3c9b97454324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:47.039996 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1366/2717] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:47.044749 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e9941-b6f2-4a79-a4ba-8f1767fe6244 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:47.113735 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e9941-b6f2-4a79-a4ba-8f1767fe6244 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:47.118841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d075e-b766-451f-a964-5a255d0a24a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:47.119472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d075e-b766-451f-a964-5a255d0a24a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:47.119749 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d075e-b766-451f-a964-5a255d0a24a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:47.120027 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d075e-b766-451f-a964-5a255d0a24a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:47.120669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d075e-b766-451f-a964-5a255d0a24a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:47.123662 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d075e-b766-451f-a964-5a255d0a24a5 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:47.162526 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1352/2718] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:47 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:47.166920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbaea99-7129-4437-88c2-3c9b97454324 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:47.180219 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbaea99-7129-4437-88c2-3c9b97454324 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:47.268719 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e1282-68e1-4053-b763-64a467e090cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:47.269663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e1282-68e1-4053-b763-64a467e090cb admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:47.270152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e1282-68e1-4053-b763-64a467e090cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:47.270669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e1282-68e1-4053-b763-64a467e090cb admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:47.271851 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e1282-68e1-4053-b763-64a467e090cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:47.275821 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e1282-68e1-4053-b763-64a467e090cb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:47.306240 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1367/2719] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:47 2022] POST /identity/v3/users => generated 295 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:47.313705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d075e-b766-451f-a964-5a255d0a24a5 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:47.328469 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d075e-b766-451f-a964-5a255d0a24a5 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:47.334769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f19d9-eea4-4054-b770-57e4679de514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:47.335412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f19d9-eea4-4054-b770-57e4679de514 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:47.335695 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f19d9-eea4-4054-b770-57e4679de514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:47.335965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f19d9-eea4-4054-b770-57e4679de514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:47.336657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f19d9-eea4-4054-b770-57e4679de514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:47.338340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f19d9-eea4-4054-b770-57e4679de514 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:47.351630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1353/2720] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:47.359351 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e1282-68e1-4053-b763-64a467e090cb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:47.370408 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e1282-68e1-4053-b763-64a467e090cb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:47.378055 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9cc5a-27f6-4997-a9aa-a758171738c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:47.378880 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9cc5a-27f6-4997-a9aa-a758171738c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:47.379268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9cc5a-27f6-4997-a9aa-a758171738c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:47.379641 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9cc5a-27f6-4997-a9aa-a758171738c9 admin admin] PATH_INFO: `/v3/projects/9bd1457ea520490daa95d4cee364c7b9/users/1164a70306d344e69fd1c2df90c52a66/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:47.398266 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9cc5a-27f6-4997-a9aa-a758171738c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd1457ea520490daa95d4cee364c7b9, user_id=1164a70306d344e69fd1c2df90c52a66, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:47.401075 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9cc5a-27f6-4997-a9aa-a758171738c9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:47.417024 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:47.428089 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1368/2721] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:47 2022] PUT /identity/v3/projects/9bd1457ea520490daa95d4cee364c7b9/users/1164a70306d344e69fd1c2df90c52a66/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:47.433962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f19d9-eea4-4054-b770-57e4679de514 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:47.451739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f19d9-eea4-4054-b770-57e4679de514 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:47.554657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39094a-cce8-4275-bcff-06d7e750cd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:47.555452 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39094a-cce8-4275-bcff-06d7e750cd37 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:47.555965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39094a-cce8-4275-bcff-06d7e750cd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:47.556456 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39094a-cce8-4275-bcff-06d7e750cd37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:47.557889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39094a-cce8-4275-bcff-06d7e750cd37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:47.561684 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39094a-cce8-4275-bcff-06d7e750cd37 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:47.566402 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:47.583740 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1354/2722] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:47.588642 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9cc5a-27f6-4997-a9aa-a758171738c9 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:47.604688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9cc5a-27f6-4997-a9aa-a758171738c9 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:47.614828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5985768-e157-4e84-a84a-dd43146d9597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:47.615972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5985768-e157-4e84-a84a-dd43146d9597 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:47.616449 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5985768-e157-4e84-a84a-dd43146d9597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:47.616930 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5985768-e157-4e84-a84a-dd43146d9597 admin admin] PATH_INFO: `/v3/projects/9bd1457ea520490daa95d4cee364c7b9/users/1164a70306d344e69fd1c2df90c52a66/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:47.620676 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5985768-e157-4e84-a84a-dd43146d9597 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd1457ea520490daa95d4cee364c7b9, user_id=1164a70306d344e69fd1c2df90c52a66, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:47.626671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5985768-e157-4e84-a84a-dd43146d9597 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:47.646296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1369/2723] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:47 2022] PUT /identity/v3/projects/9bd1457ea520490daa95d4cee364c7b9/users/1164a70306d344e69fd1c2df90c52a66/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:47.664137 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970e429-6a59-4735-b120-3023aecd53a4 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:47.664624 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970e429-6a59-4735-b120-3023aecd53a4 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:47.665158 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970e429-6a59-4735-b120-3023aecd53a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:47.716984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-c970e429-6a59-4735-b120-3023aecd53a4 None None] MFA Rules not processed for user `1164a70306d344e69fd1c2df90c52a66`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:47.718548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-c970e429-6a59-4735-b120-3023aecd53a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:47.838869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1355/2724] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:47.851194 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5985768-e157-4e84-a84a-dd43146d9597 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:47.854219 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c5985768-e157-4e84-a84a-dd43146d9597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:47.958591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5985768-e157-4e84-a84a-dd43146d9597 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:47.962966 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c3572-ea1f-4bb7-a4da-5379c39fca47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:47.963600 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c3572-ea1f-4bb7-a4da-5379c39fca47 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:47.963881 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c3572-ea1f-4bb7-a4da-5379c39fca47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:47.964314 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c3572-ea1f-4bb7-a4da-5379c39fca47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:48.041306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c3572-ea1f-4bb7-a4da-5379c39fca47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:48.042783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c3572-ea1f-4bb7-a4da-5379c39fca47 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:48.054180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1370/2725] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:49.270657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb564a-922d-44ac-b321-05dd102f8943 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:49.271208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb564a-922d-44ac-b321-05dd102f8943 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:49.271208 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb564a-922d-44ac-b321-05dd102f8943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:49.292143 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-baeb564a-922d-44ac-b321-05dd102f8943 None None] MFA Rules not processed for user `086e60f669894290be19bd9b4b40d1d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:49.293432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-baeb564a-922d-44ac-b321-05dd102f8943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:49.365342 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1356/2726] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:52.871716 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c3572-ea1f-4bb7-a4da-5379c39fca47 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:52.886135 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c3572-ea1f-4bb7-a4da-5379c39fca47 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:52.891696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86288136-066f-4c52-b4bf-22899fe345b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:52.892464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86288136-066f-4c52-b4bf-22899fe345b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:52.892747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86288136-066f-4c52-b4bf-22899fe345b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:52.893332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86288136-066f-4c52-b4bf-22899fe345b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:52.912458 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86288136-066f-4c52-b4bf-22899fe345b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:52.915169 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86288136-066f-4c52-b4bf-22899fe345b7 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:52.930099 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1371/2727] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:54.036310 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e648a09-bb92-4cc8-8997-e4f4d9cc4cda None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:54.037711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e648a09-bb92-4cc8-8997-e4f4d9cc4cda None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:54.037711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e648a09-bb92-4cc8-8997-e4f4d9cc4cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:54.059921 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-7e648a09-bb92-4cc8-8997-e4f4d9cc4cda None None] MFA Rules not processed for user `fa949e48248747369eefd74280441b75`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:54.061248 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-7e648a09-bb92-4cc8-8997-e4f4d9cc4cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:54.128074 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1357/2728] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:54.703556 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86288136-066f-4c52-b4bf-22899fe345b7 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:54.705601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-86288136-066f-4c52-b4bf-22899fe345b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:54.827607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86288136-066f-4c52-b4bf-22899fe345b7 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:54.832619 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7baea-4436-4395-afb5-c23606558da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:54.833611 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7baea-4436-4395-afb5-c23606558da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:54.834039 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7baea-4436-4395-afb5-c23606558da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:54.834447 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7baea-4436-4395-afb5-c23606558da7 admin admin] PATH_INFO: `/v3/users/992b881e8bb04aa1a30a5d3ff0f1af5b` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:54.836466 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7baea-4436-4395-afb5-c23606558da7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=992b881e8bb04aa1a30a5d3ff0f1af5b)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:54.839877 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7baea-4436-4395-afb5-c23606558da7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:54.882899 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:54.925574 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bef7baea-4436-4395-afb5-c23606558da7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '992b881e8bb04aa1a30a5d3ff0f1af5b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:54.949788 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bef7baea-4436-4395-afb5-c23606558da7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '992b881e8bb04aa1a30a5d3ff0f1af5b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:54.968652 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bef7baea-4436-4395-afb5-c23606558da7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '992b881e8bb04aa1a30a5d3ff0f1af5b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:54.969564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bef7baea-4436-4395-afb5-c23606558da7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '992b881e8bb04aa1a30a5d3ff0f1af5b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:54.979814 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-bef7baea-4436-4395-afb5-c23606558da7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '992b881e8bb04aa1a30a5d3ff0f1af5b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:54.991658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1372/2729] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:54 2022] DELETE /identity/v3/users/992b881e8bb04aa1a30a5d3ff0f1af5b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:54.994088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa3879-44ad-40fa-94d9-7e4abc413b66 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:54.994377 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa3879-44ad-40fa-94d9-7e4abc413b66 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:54.994657 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa3879-44ad-40fa-94d9-7e4abc413b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:55.019410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-80fa3879-44ad-40fa-94d9-7e4abc413b66 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:55.020628 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-80fa3879-44ad-40fa-94d9-7e4abc413b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:55.058763 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:55.111025 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1358/2730] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:55.125298 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7baea-4436-4395-afb5-c23606558da7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:55.127254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bef7baea-4436-4395-afb5-c23606558da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:55.247048 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7baea-4436-4395-afb5-c23606558da7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:55.251475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e0347-60e0-4788-b278-3eb9c8e78519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:55.252224 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e0347-60e0-4788-b278-3eb9c8e78519 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:55.252572 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e0347-60e0-4788-b278-3eb9c8e78519 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:55.252980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e0347-60e0-4788-b278-3eb9c8e78519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:55.266327 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e0347-60e0-4788-b278-3eb9c8e78519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:55.267898 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e0347-60e0-4788-b278-3eb9c8e78519 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:55.280794 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1373/2731] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:55.774719 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa3879-44ad-40fa-94d9-7e4abc413b66 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:55.777571 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-80fa3879-44ad-40fa-94d9-7e4abc413b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:55.804864 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa3879-44ad-40fa-94d9-7e4abc413b66 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:55.824262 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da535e-3a8d-42ab-80b3-bff2a287a3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:55.824891 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da535e-3a8d-42ab-80b3-bff2a287a3b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:55.825847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da535e-3a8d-42ab-80b3-bff2a287a3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:55.826124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da535e-3a8d-42ab-80b3-bff2a287a3b4 admin admin] PATH_INFO: `/v3/projects/c468621fa4aa429aab72886bfcf95ee3` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:55.827911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da535e-3a8d-42ab-80b3-bff2a287a3b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c468621fa4aa429aab72886bfcf95ee3)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:55.830453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da535e-3a8d-42ab-80b3-bff2a287a3b4 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:55.939856 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-95da535e-3a8d-42ab-80b3-bff2a287a3b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c468621fa4aa429aab72886bfcf95ee3', 'request_id': 'req-95da535e-3a8d-42ab-80b3-bff2a287a3b4'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:55.951189 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-95da535e-3a8d-42ab-80b3-bff2a287a3b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c468621fa4aa429aab72886bfcf95ee3', 'request_id': 'req-95da535e-3a8d-42ab-80b3-bff2a287a3b4'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:55.971400 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-95da535e-3a8d-42ab-80b3-bff2a287a3b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c468621fa4aa429aab72886bfcf95ee3', 'request_id': 'req-95da535e-3a8d-42ab-80b3-bff2a287a3b4'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:55.993492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-95da535e-3a8d-42ab-80b3-bff2a287a3b4 admin admin] The token cache is being invalidate because project c468621fa4aa429aab72886bfcf95ee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:55.993788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-95da535e-3a8d-42ab-80b3-bff2a287a3b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:55.994508 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-95da535e-3a8d-42ab-80b3-bff2a287a3b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:55.996386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1359/2732] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:55 2022] DELETE /identity/v3/projects/c468621fa4aa429aab72886bfcf95ee3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:05.274332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=45,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:05.989261 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:15.503293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e0347-60e0-4788-b278-3eb9c8e78519 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:15.505479 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-582e0347-60e0-4788-b278-3eb9c8e78519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:15.526309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e0347-60e0-4788-b278-3eb9c8e78519 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:15.626835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feeb6d9-d689-4b7c-9cf6-dd7113b392fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:15.627582 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feeb6d9-d689-4b7c-9cf6-dd7113b392fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:15.627924 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feeb6d9-d689-4b7c-9cf6-dd7113b392fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:15.628273 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feeb6d9-d689-4b7c-9cf6-dd7113b392fb admin admin] PATH_INFO: `/v3/users/876874d9ff5b4a9b8248f4ef6366430e` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:15.629998 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feeb6d9-d689-4b7c-9cf6-dd7113b392fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=876874d9ff5b4a9b8248f4ef6366430e)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:15.634353 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feeb6d9-d689-4b7c-9cf6-dd7113b392fb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:15.731885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0feeb6d9-d689-4b7c-9cf6-dd7113b392fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '876874d9ff5b4a9b8248f4ef6366430e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:15.752316 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0feeb6d9-d689-4b7c-9cf6-dd7113b392fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '876874d9ff5b4a9b8248f4ef6366430e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:15.763980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0feeb6d9-d689-4b7c-9cf6-dd7113b392fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '876874d9ff5b4a9b8248f4ef6366430e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:15.765514 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0feeb6d9-d689-4b7c-9cf6-dd7113b392fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '876874d9ff5b4a9b8248f4ef6366430e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:15.774907 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0feeb6d9-d689-4b7c-9cf6-dd7113b392fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '876874d9ff5b4a9b8248f4ef6366430e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:15.784221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1374/2733] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:15 2022] DELETE /identity/v3/users/876874d9ff5b4a9b8248f4ef6366430e => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:15.790927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97307284-00e9-4d43-b509-970152bbc483 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:15.791756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97307284-00e9-4d43-b509-970152bbc483 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:15.791756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97307284-00e9-4d43-b509-970152bbc483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:15.816901 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-97307284-00e9-4d43-b509-970152bbc483 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:15.818254 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-97307284-00e9-4d43-b509-970152bbc483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:15.895332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4a9a1-dbfd-4355-b51c-3a3341121bc1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:15.895632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4a9a1-dbfd-4355-b51c-3a3341121bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:15.895893 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4a9a1-dbfd-4355-b51c-3a3341121bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:15.914422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1360/2734] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:15.920286 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-75a4a9a1-dbfd-4355-b51c-3a3341121bc1 None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:15.921614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-75a4a9a1-dbfd-4355-b51c-3a3341121bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:15.928516 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97307284-00e9-4d43-b509-970152bbc483 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:15.930432 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-97307284-00e9-4d43-b509-970152bbc483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:16.009339 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97307284-00e9-4d43-b509-970152bbc483 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:16.014013 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f059d-e554-4b88-b75b-bd50fbc44400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:16.014290 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1375/2735] 10.222.0.88 () {58 vars in 1009 bytes} [Tue Aug 23 22:34:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:16.014941 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f059d-e554-4b88-b75b-bd50fbc44400 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:16.015837 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f059d-e554-4b88-b75b-bd50fbc44400 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:16.016229 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f059d-e554-4b88-b75b-bd50fbc44400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:16.031091 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f059d-e554-4b88-b75b-bd50fbc44400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:16.034071 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f059d-e554-4b88-b75b-bd50fbc44400 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:16.051877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1361/2736] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:34:16.085495 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4a9a1-dbfd-4355-b51c-3a3341121bc1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:16.087462 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-75a4a9a1-dbfd-4355-b51c-3a3341121bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:16.131274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:16.221444 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4a9a1-dbfd-4355-b51c-3a3341121bc1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:16.229979 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e211ab0-905b-46e9-b802-696b45ee0061 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:16.231135 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e211ab0-905b-46e9-b802-696b45ee0061 service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:16.232124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e211ab0-905b-46e9-b802-696b45ee0061 service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:16.232124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e211ab0-905b-46e9-b802-696b45ee0061 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:16.248173 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e211ab0-905b-46e9-b802-696b45ee0061 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:16.250563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e211ab0-905b-46e9-b802-696b45ee0061 service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:16.273653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1376/2737] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:34:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:34:16.495694 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f059d-e554-4b88-b75b-bd50fbc44400 service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:16.498356 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-9a5f059d-e554-4b88-b75b-bd50fbc44400 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:16.522889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f059d-e554-4b88-b75b-bd50fbc44400 service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:16.531076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d50631-83f5-4a5f-b260-de85efb104da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:16.532213 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d50631-83f5-4a5f-b260-de85efb104da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:16.532774 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d50631-83f5-4a5f-b260-de85efb104da admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:16.533381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d50631-83f5-4a5f-b260-de85efb104da admin admin] PATH_INFO: `/v3/projects/c13d31d53cd747308f43e27a32f279a7` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:16.535651 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d50631-83f5-4a5f-b260-de85efb104da admin admin] RBAC: Authorizing `identity:delete_project(project_id=c13d31d53cd747308f43e27a32f279a7)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:16.539820 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d50631-83f5-4a5f-b260-de85efb104da admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:16.622370 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-36d50631-83f5-4a5f-b260-de85efb104da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c13d31d53cd747308f43e27a32f279a7', 'request_id': 'req-36d50631-83f5-4a5f-b260-de85efb104da'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:16.633275 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-36d50631-83f5-4a5f-b260-de85efb104da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c13d31d53cd747308f43e27a32f279a7', 'request_id': 'req-36d50631-83f5-4a5f-b260-de85efb104da'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:16.643705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-36d50631-83f5-4a5f-b260-de85efb104da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c13d31d53cd747308f43e27a32f279a7', 'request_id': 'req-36d50631-83f5-4a5f-b260-de85efb104da'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:16.672494 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-36d50631-83f5-4a5f-b260-de85efb104da admin admin] The token cache is being invalidate because project c13d31d53cd747308f43e27a32f279a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:16.673088 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-36d50631-83f5-4a5f-b260-de85efb104da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:16.674467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-36d50631-83f5-4a5f-b260-de85efb104da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:16.678285 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1362/2738] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:16 2022] DELETE /identity/v3/projects/c13d31d53cd747308f43e27a32f279a7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:16.684281 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e211ab0-905b-46e9-b802-696b45ee0061 service swift] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:16.708056 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e211ab0-905b-46e9-b802-696b45ee0061 service swift] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:16.864151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31af04a-9cd0-4dbe-a9ab-12990892c9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:16.865164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31af04a-9cd0-4dbe-a9ab-12990892c9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:16.865634 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31af04a-9cd0-4dbe-a9ab-12990892c9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:16.866111 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31af04a-9cd0-4dbe-a9ab-12990892c9fd admin admin] PATH_INFO: `/v3/users/1164a70306d344e69fd1c2df90c52a66` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:16.868300 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31af04a-9cd0-4dbe-a9ab-12990892c9fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1164a70306d344e69fd1c2df90c52a66)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:16.872604 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31af04a-9cd0-4dbe-a9ab-12990892c9fd admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:16.987474 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d31af04a-9cd0-4dbe-a9ab-12990892c9fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1164a70306d344e69fd1c2df90c52a66'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:17.015747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d31af04a-9cd0-4dbe-a9ab-12990892c9fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1164a70306d344e69fd1c2df90c52a66'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:17.029717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d31af04a-9cd0-4dbe-a9ab-12990892c9fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1164a70306d344e69fd1c2df90c52a66'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:17.030650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d31af04a-9cd0-4dbe-a9ab-12990892c9fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1164a70306d344e69fd1c2df90c52a66'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:17.040988 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d31af04a-9cd0-4dbe-a9ab-12990892c9fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1164a70306d344e69fd1c2df90c52a66'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:17.051167 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1377/2739] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:16 2022] DELETE /identity/v3/users/1164a70306d344e69fd1c2df90c52a66 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:17.150922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bfba3-eb22-4398-ab2c-c3341685672f None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:17.151760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bfba3-eb22-4398-ab2c-c3341685672f None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:17.151760 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bfba3-eb22-4398-ab2c-c3341685672f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:17.176723 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-726bfba3-eb22-4398-ab2c-c3341685672f None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:17.178482 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-726bfba3-eb22-4398-ab2c-c3341685672f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:17.211878 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:17.324496 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1363/2740] 10.222.0.88 () {58 vars in 967 bytes} [Tue Aug 23 22:34:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:17.339523 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31af04a-9cd0-4dbe-a9ab-12990892c9fd admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:17.342764 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d31af04a-9cd0-4dbe-a9ab-12990892c9fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:17.421027 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:17.496839 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31af04a-9cd0-4dbe-a9ab-12990892c9fd admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:17.503645 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fe60e-1e01-401c-9a7e-9792bf7a934d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:17.504322 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fe60e-1e01-401c-9a7e-9792bf7a934d service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:17.504713 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fe60e-1e01-401c-9a7e-9792bf7a934d service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:17.505222 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fe60e-1e01-401c-9a7e-9792bf7a934d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:17.508384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bfba3-eb22-4398-ab2c-c3341685672f None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:17.510028 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-726bfba3-eb22-4398-ab2c-c3341685672f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:17.520001 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46fe60e-1e01-401c-9a7e-9792bf7a934d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:17.522138 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46fe60e-1e01-401c-9a7e-9792bf7a934d service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:17.537420 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1378/2741] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:34:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:34:17.636100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bfba3-eb22-4398-ab2c-c3341685672f None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:17.641758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a97ce2-1d96-470e-8d13-d2dc636b6838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:17.642421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a97ce2-1d96-470e-8d13-d2dc636b6838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:17.642700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a97ce2-1d96-470e-8d13-d2dc636b6838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:17.642983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a97ce2-1d96-470e-8d13-d2dc636b6838 admin admin] PATH_INFO: `/v3/projects/9bd1457ea520490daa95d4cee364c7b9` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:17.644314 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a97ce2-1d96-470e-8d13-d2dc636b6838 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd1457ea520490daa95d4cee364c7b9)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:17.646569 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a97ce2-1d96-470e-8d13-d2dc636b6838 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:17.722567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f3a97ce2-1d96-470e-8d13-d2dc636b6838 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd1457ea520490daa95d4cee364c7b9', 'request_id': 'req-f3a97ce2-1d96-470e-8d13-d2dc636b6838'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:17.732329 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f3a97ce2-1d96-470e-8d13-d2dc636b6838 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd1457ea520490daa95d4cee364c7b9', 'request_id': 'req-f3a97ce2-1d96-470e-8d13-d2dc636b6838'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:17.740859 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f3a97ce2-1d96-470e-8d13-d2dc636b6838 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd1457ea520490daa95d4cee364c7b9', 'request_id': 'req-f3a97ce2-1d96-470e-8d13-d2dc636b6838'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:17.774775 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f3a97ce2-1d96-470e-8d13-d2dc636b6838 admin admin] The token cache is being invalidate because project 9bd1457ea520490daa95d4cee364c7b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:17.775416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f3a97ce2-1d96-470e-8d13-d2dc636b6838 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:17.776661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-f3a97ce2-1d96-470e-8d13-d2dc636b6838 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:17.780512 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1364/2742] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:17 2022] DELETE /identity/v3/projects/9bd1457ea520490daa95d4cee364c7b9 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:34:17.786438 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eb5ed-a19d-4933-8b2b-f4a79c2eed75 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:17.787302 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eb5ed-a19d-4933-8b2b-f4a79c2eed75 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:17.787302 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eb5ed-a19d-4933-8b2b-f4a79c2eed75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:17.810903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-486eb5ed-a19d-4933-8b2b-f4a79c2eed75 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:17.812190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-486eb5ed-a19d-4933-8b2b-f4a79c2eed75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:17.877942 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42118c26-b3ee-44a5-b995-973b8115b178 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:17.878243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42118c26-b3ee-44a5-b995-973b8115b178 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:17.878533 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42118c26-b3ee-44a5-b995-973b8115b178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:17.906558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1379/2743] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:17.908825 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-42118c26-b3ee-44a5-b995-973b8115b178 None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:17.910144 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-42118c26-b3ee-44a5-b995-973b8115b178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:17.921154 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a915d-ae77-4ebf-882c-1379339fea6c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:17.921807 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a915d-ae77-4ebf-882c-1379339fea6c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:17.922148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a915d-ae77-4ebf-882c-1379339fea6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:17.959607 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-296a915d-ae77-4ebf-882c-1379339fea6c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:17.960875 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-296a915d-ae77-4ebf-882c-1379339fea6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:17.970430 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1380/2744] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:17.975795 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a915d-ae77-4ebf-882c-1379339fea6c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:17.988567 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a915d-ae77-4ebf-882c-1379339fea6c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:17.995540 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0511280-94bb-47cc-8069-f95da22a8fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:17.997435 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0511280-94bb-47cc-8069-f95da22a8fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:17.998164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1365/2745] 10.222.0.88 () {58 vars in 1009 bytes} [Tue Aug 23 22:34:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:18.001635 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0511280-94bb-47cc-8069-f95da22a8fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:18.002100 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0511280-94bb-47cc-8069-f95da22a8fb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:18.003085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0511280-94bb-47cc-8069-f95da22a8fb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:34:18.003483 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0511280-94bb-47cc-8069-f95da22a8fb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:18.005436 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0511280-94bb-47cc-8069-f95da22a8fb0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:18.016668 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42118c26-b3ee-44a5-b995-973b8115b178 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:18.018338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-42118c26-b3ee-44a5-b995-973b8115b178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:18.026183 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1381/2746] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:34:18.031678 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0511280-94bb-47cc-8069-f95da22a8fb0 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:18.035326 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:18.045270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0511280-94bb-47cc-8069-f95da22a8fb0 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:18.052708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc28e35-7111-45a1-97f2-fdb92762e41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:18.053347 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc28e35-7111-45a1-97f2-fdb92762e41c admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:18.054247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc28e35-7111-45a1-97f2-fdb92762e41c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:18.054247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc28e35-7111-45a1-97f2-fdb92762e41c admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:18.054618 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc28e35-7111-45a1-97f2-fdb92762e41c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:18.057001 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc28e35-7111-45a1-97f2-fdb92762e41c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:18.111626 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1382/2747] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:18 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:34:18.116780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc28e35-7111-45a1-97f2-fdb92762e41c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:18.164472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42118c26-b3ee-44a5-b995-973b8115b178 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:18.217802 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc28e35-7111-45a1-97f2-fdb92762e41c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:18.225683 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f0eb0-dc5d-412a-a269-f3dec638428b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:18.226663 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f0eb0-dc5d-412a-a269-f3dec638428b admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:18.227115 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f0eb0-dc5d-412a-a269-f3dec638428b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:18.227565 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f0eb0-dc5d-412a-a269-f3dec638428b admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:18.228643 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f0eb0-dc5d-412a-a269-f3dec638428b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:18.232443 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f0eb0-dc5d-412a-a269-f3dec638428b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:18.269750 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1383/2748] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:18 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:18.274810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f0eb0-dc5d-412a-a269-f3dec638428b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:18.278786 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab7732-bcdd-459f-b1fd-957d043c4512 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:18.279965 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab7732-bcdd-459f-b1fd-957d043c4512 service swift] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:18.280484 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab7732-bcdd-459f-b1fd-957d043c4512 service swift] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:18.281004 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab7732-bcdd-459f-b1fd-957d043c4512 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:18.282238 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:18.301088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f0eb0-dc5d-412a-a269-f3dec638428b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:18.301088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7b14c-a67b-4ed2-8cea-4ee450238ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:18.301088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7b14c-a67b-4ed2-8cea-4ee450238ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:18.301088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7b14c-a67b-4ed2-8cea-4ee450238ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:18.301854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7b14c-a67b-4ed2-8cea-4ee450238ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:18.301854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7b14c-a67b-4ed2-8cea-4ee450238ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:18.302564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7b14c-a67b-4ed2-8cea-4ee450238ea7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:18.317592 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1384/2749] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:18.322149 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7b14c-a67b-4ed2-8cea-4ee450238ea7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:18.337303 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7b14c-a67b-4ed2-8cea-4ee450238ea7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:18.345093 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f136c-3b19-409b-8b77-b7d224725672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:18.345729 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f136c-3b19-409b-8b77-b7d224725672 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:18.346037 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f136c-3b19-409b-8b77-b7d224725672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:18.346317 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f136c-3b19-409b-8b77-b7d224725672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:18.346963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f136c-3b19-409b-8b77-b7d224725672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:18.348707 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f136c-3b19-409b-8b77-b7d224725672 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:18.372277 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1385/2750] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:34:18.377225 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f136c-3b19-409b-8b77-b7d224725672 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:18.394641 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f136c-3b19-409b-8b77-b7d224725672 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:18.400829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab7732-bcdd-459f-b1fd-957d043c4512 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:18.401611 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb6f6f-b555-432a-8ef0-d096afb09f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:18.402657 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb6f6f-b555-432a-8ef0-d096afb09f05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:18.403088 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb6f6f-b555-432a-8ef0-d096afb09f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:18.403530 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb6f6f-b555-432a-8ef0-d096afb09f05 admin admin] PATH_INFO: `/v3/projects/e83d9d847694443889377eefa24f7feb/users/bff3dd9bc1c34e61b7d061ff2ac6626a/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:18.404375 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab7732-bcdd-459f-b1fd-957d043c4512 service swift] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:18.421313 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1366/2751] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:34:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 22:34:18.432464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb6f6f-b555-432a-8ef0-d096afb09f05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e83d9d847694443889377eefa24f7feb, user_id=bff3dd9bc1c34e61b7d061ff2ac6626a, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:18.436958 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb6f6f-b555-432a-8ef0-d096afb09f05 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:18.464677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1386/2752] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:18 2022] PUT /identity/v3/projects/e83d9d847694443889377eefa24f7feb/users/bff3dd9bc1c34e61b7d061ff2ac6626a/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:34:18.477005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d99a7-a723-4129-8c2d-d8356e124961 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:18.477379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d99a7-a723-4129-8c2d-d8356e124961 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:18.477627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d99a7-a723-4129-8c2d-d8356e124961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:18.517306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-f41d99a7-a723-4129-8c2d-d8356e124961 None None] MFA Rules not processed for user `bff3dd9bc1c34e61b7d061ff2ac6626a`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:18.519346 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f41d99a7-a723-4129-8c2d-d8356e124961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:18.579710 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:18.622636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1367/2753] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:18.632456 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb6f6f-b555-432a-8ef0-d096afb09f05 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:18.743381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb6f6f-b555-432a-8ef0-d096afb09f05 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:18.753933 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900afcc5-734c-4805-b594-07f3728d48aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:18.754708 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900afcc5-734c-4805-b594-07f3728d48aa admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:18.755066 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900afcc5-734c-4805-b594-07f3728d48aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:18.755330 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900afcc5-734c-4805-b594-07f3728d48aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:18.755967 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900afcc5-734c-4805-b594-07f3728d48aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:18.758522 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900afcc5-734c-4805-b594-07f3728d48aa admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:18.799964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1387/2754] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:18 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:18.805593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d99a7-a723-4129-8c2d-d8356e124961 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:18.825329 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d99a7-a723-4129-8c2d-d8356e124961 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:18.918781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3394ea-085d-4e8b-ae9b-10ba6a89ff46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:18.919682 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3394ea-085d-4e8b-ae9b-10ba6a89ff46 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:18.920203 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3394ea-085d-4e8b-ae9b-10ba6a89ff46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:18.920677 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3394ea-085d-4e8b-ae9b-10ba6a89ff46 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:18.923005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3394ea-085d-4e8b-ae9b-10ba6a89ff46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:18.926069 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3394ea-085d-4e8b-ae9b-10ba6a89ff46 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:18.926998 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576f4c2-eda6-40fd-aeda-d4cf7d9ecbc0 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:18.927333 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576f4c2-eda6-40fd-aeda-d4cf7d9ecbc0 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:18.927630 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576f4c2-eda6-40fd-aeda-d4cf7d9ecbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:18.954068 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7576f4c2-eda6-40fd-aeda-d4cf7d9ecbc0 None None] MFA Rules not processed for user `cb001b14eec54047aee42ca6bd4a1eba`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:18.955325 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7576f4c2-eda6-40fd-aeda-d4cf7d9ecbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:18.973129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1368/2755] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:18 2022] POST /identity/v3/users => generated 299 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:18.974401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3394ea-085d-4e8b-ae9b-10ba6a89ff46 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:18.996961 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3394ea-085d-4e8b-ae9b-10ba6a89ff46 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:19.005268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f493be2-611b-421c-a1c3-3a86052a7b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:19.005545 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f493be2-611b-421c-a1c3-3a86052a7b8c admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:19.005850 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f493be2-611b-421c-a1c3-3a86052a7b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:19.008338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f493be2-611b-421c-a1c3-3a86052a7b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:19.008338 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f493be2-611b-421c-a1c3-3a86052a7b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:19.008781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f493be2-611b-421c-a1c3-3a86052a7b8c admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:19.030724 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1369/2756] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:19.035301 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f493be2-611b-421c-a1c3-3a86052a7b8c admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:19.051840 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f493be2-611b-421c-a1c3-3a86052a7b8c admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:19.060773 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6a23f-3c84-49ed-87f7-dd721c7a8db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:19.062198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6a23f-3c84-49ed-87f7-dd721c7a8db0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:19.062613 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6a23f-3c84-49ed-87f7-dd721c7a8db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:19.063043 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6a23f-3c84-49ed-87f7-dd721c7a8db0 admin admin] PATH_INFO: `/v3/projects/03c88f56ad194520a936a77c7afde3e2/users/0f14f9e8bb7f44559d50bfadd1e26669/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:19.074176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1388/2757] 10.222.0.88 () {58 vars in 967 bytes} [Tue Aug 23 22:34:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:19.090555 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6a23f-3c84-49ed-87f7-dd721c7a8db0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03c88f56ad194520a936a77c7afde3e2, user_id=0f14f9e8bb7f44559d50bfadd1e26669, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:19.094551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576f4c2-eda6-40fd-aeda-d4cf7d9ecbc0 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:19.096467 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6a23f-3c84-49ed-87f7-dd721c7a8db0 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:19.125283 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1370/2758] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:19 2022] PUT /identity/v3/projects/03c88f56ad194520a936a77c7afde3e2/users/0f14f9e8bb7f44559d50bfadd1e26669/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:34:19.131624 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6a23f-3c84-49ed-87f7-dd721c7a8db0 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:19.190293 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:19.231213 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576f4c2-eda6-40fd-aeda-d4cf7d9ecbc0 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:19.256880 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6a23f-3c84-49ed-87f7-dd721c7a8db0 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:19.265923 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd04db1-3e31-4cb8-8b88-2009c0c983ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:19.266975 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd04db1-3e31-4cb8-8b88-2009c0c983ec admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:19.267459 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd04db1-3e31-4cb8-8b88-2009c0c983ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:19.267917 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd04db1-3e31-4cb8-8b88-2009c0c983ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:19.269121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd04db1-3e31-4cb8-8b88-2009c0c983ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:19.272565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd04db1-3e31-4cb8-8b88-2009c0c983ec admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:19.302877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1371/2759] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:19.308227 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd04db1-3e31-4cb8-8b88-2009c0c983ec admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:19.330359 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd04db1-3e31-4cb8-8b88-2009c0c983ec admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:19.339026 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62914e07-cd0b-4635-b03b-59d5525bf15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:19.340842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62914e07-cd0b-4635-b03b-59d5525bf15d admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:19.340842 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62914e07-cd0b-4635-b03b-59d5525bf15d admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:19.341424 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62914e07-cd0b-4635-b03b-59d5525bf15d admin admin] PATH_INFO: `/v3/projects/03c88f56ad194520a936a77c7afde3e2/users/0f14f9e8bb7f44559d50bfadd1e26669/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:19.342510 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb2caa-7dc1-4820-875f-6aaa471b32cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601a3defca70428689ee22517d7bcb19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:19.343267 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb2caa-7dc1-4820-875f-6aaa471b32cd service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:19.343555 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb2caa-7dc1-4820-875f-6aaa471b32cd service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:19.343938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb2caa-7dc1-4820-875f-6aaa471b32cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:19.345187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62914e07-cd0b-4635-b03b-59d5525bf15d admin admin] RBAC: Authorizing `identity:create_grant(project_id=03c88f56ad194520a936a77c7afde3e2, user_id=0f14f9e8bb7f44559d50bfadd1e26669, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:19.349975 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62914e07-cd0b-4635-b03b-59d5525bf15d admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:19.376981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1372/2760] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:19 2022] PUT /identity/v3/projects/03c88f56ad194520a936a77c7afde3e2/users/0f14f9e8bb7f44559d50bfadd1e26669/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:19.385292 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf06677-da9b-405a-8f99-8fb0ceb8e2d7 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:19.385739 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf06677-da9b-405a-8f99-8fb0ceb8e2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:19.386210 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf06677-da9b-405a-8f99-8fb0ceb8e2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:19.439847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-5cf06677-da9b-405a-8f99-8fb0ceb8e2d7 None None] MFA Rules not processed for user `0f14f9e8bb7f44559d50bfadd1e26669`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:19.441981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-5cf06677-da9b-405a-8f99-8fb0ceb8e2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:19.453336 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:19.475235 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb2caa-7dc1-4820-875f-6aaa471b32cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:19.478545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb2caa-7dc1-4820-875f-6aaa471b32cd service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:19.498539 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1389/2761] 10.222.0.88 () {60 vars in 1384 bytes} [Tue Aug 23 22:34:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 22:34:19.585021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1373/2762] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:19.607023 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb2caa-7dc1-4820-875f-6aaa471b32cd service swift] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:19.609974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-eeeb2caa-7dc1-4820-875f-6aaa471b32cd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:19.721389 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb2caa-7dc1-4820-875f-6aaa471b32cd service swift] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:19.804732 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:19.813504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06764b-77ec-4eb5-81d7-9e103982e881 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:19.815693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06764b-77ec-4eb5-81d7-9e103982e881 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:19.815693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06764b-77ec-4eb5-81d7-9e103982e881 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:19.815693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06764b-77ec-4eb5-81d7-9e103982e881 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:19.827761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06764b-77ec-4eb5-81d7-9e103982e881 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:19.829393 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06764b-77ec-4eb5-81d7-9e103982e881 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:19.844435 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1390/2763] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:34:19.939610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf06677-da9b-405a-8f99-8fb0ceb8e2d7 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:20.079754 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf06677-da9b-405a-8f99-8fb0ceb8e2d7 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:20.093497 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260dc540-4988-4ef5-a282-aebd90916481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:20.094247 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dc540-4988-4ef5-a282-aebd90916481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:20.095275 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dc540-4988-4ef5-a282-aebd90916481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:20.096890 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dc540-4988-4ef5-a282-aebd90916481 admin admin] PATH_INFO: `/v3/users/bff3dd9bc1c34e61b7d061ff2ac6626a` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:20.098656 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260dc540-4988-4ef5-a282-aebd90916481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bff3dd9bc1c34e61b7d061ff2ac6626a)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:20.102731 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260dc540-4988-4ef5-a282-aebd90916481 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:20.213767 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-260dc540-4988-4ef5-a282-aebd90916481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bff3dd9bc1c34e61b7d061ff2ac6626a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:20.215148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-260dc540-4988-4ef5-a282-aebd90916481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bff3dd9bc1c34e61b7d061ff2ac6626a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:20.225493 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-260dc540-4988-4ef5-a282-aebd90916481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bff3dd9bc1c34e61b7d061ff2ac6626a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:20.243039 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-260dc540-4988-4ef5-a282-aebd90916481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bff3dd9bc1c34e61b7d061ff2ac6626a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:20.256175 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-260dc540-4988-4ef5-a282-aebd90916481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bff3dd9bc1c34e61b7d061ff2ac6626a'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:20.268174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1374/2764] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:19 2022] DELETE /identity/v3/users/bff3dd9bc1c34e61b7d061ff2ac6626a => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:20.274698 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c63d1-5a2d-4cbc-82df-afe48297c5ef None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:20.275230 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c63d1-5a2d-4cbc-82df-afe48297c5ef None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:20.275734 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c63d1-5a2d-4cbc-82df-afe48297c5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:20.311171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9d0c63d1-5a2d-4cbc-82df-afe48297c5ef None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:20.313502 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9d0c63d1-5a2d-4cbc-82df-afe48297c5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:20.460650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1391/2765] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:20.472638 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260dc540-4988-4ef5-a282-aebd90916481 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:20.475124 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-260dc540-4988-4ef5-a282-aebd90916481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:20.532206 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:20.611009 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260dc540-4988-4ef5-a282-aebd90916481 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:20.617667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301687d6-a19b-444e-b21f-acaeb0f3e435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:20.618686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301687d6-a19b-444e-b21f-acaeb0f3e435 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:20.619148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301687d6-a19b-444e-b21f-acaeb0f3e435 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:20.619606 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301687d6-a19b-444e-b21f-acaeb0f3e435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:20.641450 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301687d6-a19b-444e-b21f-acaeb0f3e435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:20.645045 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301687d6-a19b-444e-b21f-acaeb0f3e435 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:20.669101 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1375/2766] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:34:21.116263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c63d1-5a2d-4cbc-82df-afe48297c5ef None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:21.117782 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9d0c63d1-5a2d-4cbc-82df-afe48297c5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:21.158593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c63d1-5a2d-4cbc-82df-afe48297c5ef None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:21.170936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4060d-f4d9-4814-8494-1ce0abc50c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:21.171763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4060d-f4d9-4814-8494-1ce0abc50c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:21.172143 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4060d-f4d9-4814-8494-1ce0abc50c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:21.172520 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4060d-f4d9-4814-8494-1ce0abc50c38 admin admin] PATH_INFO: `/v3/projects/e83d9d847694443889377eefa24f7feb` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:21.178375 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4060d-f4d9-4814-8494-1ce0abc50c38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e83d9d847694443889377eefa24f7feb)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:21.182651 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4060d-f4d9-4814-8494-1ce0abc50c38 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:21.277885 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4dd4060d-f4d9-4814-8494-1ce0abc50c38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e83d9d847694443889377eefa24f7feb', 'request_id': 'req-4dd4060d-f4d9-4814-8494-1ce0abc50c38'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:21.287791 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4dd4060d-f4d9-4814-8494-1ce0abc50c38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e83d9d847694443889377eefa24f7feb', 'request_id': 'req-4dd4060d-f4d9-4814-8494-1ce0abc50c38'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:21.297892 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4dd4060d-f4d9-4814-8494-1ce0abc50c38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e83d9d847694443889377eefa24f7feb', 'request_id': 'req-4dd4060d-f4d9-4814-8494-1ce0abc50c38'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:21.313148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4dd4060d-f4d9-4814-8494-1ce0abc50c38 admin admin] The token cache is being invalidate because project e83d9d847694443889377eefa24f7feb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:21.313487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4dd4060d-f4d9-4814-8494-1ce0abc50c38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:21.314216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4dd4060d-f4d9-4814-8494-1ce0abc50c38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:21.316112 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1392/2767] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:21 2022] DELETE /identity/v3/projects/e83d9d847694443889377eefa24f7feb => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:34:21.320554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301687d6-a19b-444e-b21f-acaeb0f3e435 service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:21.332656 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301687d6-a19b-444e-b21f-acaeb0f3e435 service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:21.455025 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601b254-2771-41bd-8ccd-88e8628f0e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:21.457234 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601b254-2771-41bd-8ccd-88e8628f0e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:21.457417 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601b254-2771-41bd-8ccd-88e8628f0e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:21.457829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601b254-2771-41bd-8ccd-88e8628f0e6b admin admin] PATH_INFO: `/v3/users/0f14f9e8bb7f44559d50bfadd1e26669` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:21.459945 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601b254-2771-41bd-8ccd-88e8628f0e6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f14f9e8bb7f44559d50bfadd1e26669)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:21.463595 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601b254-2771-41bd-8ccd-88e8628f0e6b admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:21.560957 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8601b254-2771-41bd-8ccd-88e8628f0e6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f14f9e8bb7f44559d50bfadd1e26669'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:21.561997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8601b254-2771-41bd-8ccd-88e8628f0e6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f14f9e8bb7f44559d50bfadd1e26669'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:21.574921 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8601b254-2771-41bd-8ccd-88e8628f0e6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f14f9e8bb7f44559d50bfadd1e26669'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:21.598541 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8601b254-2771-41bd-8ccd-88e8628f0e6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f14f9e8bb7f44559d50bfadd1e26669'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:21.612991 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8601b254-2771-41bd-8ccd-88e8628f0e6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f14f9e8bb7f44559d50bfadd1e26669'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:21.626393 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1376/2768] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:21 2022] DELETE /identity/v3/users/0f14f9e8bb7f44559d50bfadd1e26669 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:34:22.223615 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4060d-f4d9-4814-8494-1ce0abc50c38 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:22.226964 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4dd4060d-f4d9-4814-8494-1ce0abc50c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:22.264006 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4060d-f4d9-4814-8494-1ce0abc50c38 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:22.350356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54458e67-7d99-476e-930c-75848eb2c1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:22.351018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54458e67-7d99-476e-930c-75848eb2c1b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:22.351296 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54458e67-7d99-476e-930c-75848eb2c1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:22.351658 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54458e67-7d99-476e-930c-75848eb2c1b4 admin admin] PATH_INFO: `/v3/projects/03c88f56ad194520a936a77c7afde3e2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:22.353082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54458e67-7d99-476e-930c-75848eb2c1b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03c88f56ad194520a936a77c7afde3e2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:22.355602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54458e67-7d99-476e-930c-75848eb2c1b4 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:22.379399 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:22.433931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-54458e67-7d99-476e-930c-75848eb2c1b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03c88f56ad194520a936a77c7afde3e2', 'request_id': 'req-54458e67-7d99-476e-930c-75848eb2c1b4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:22.441854 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-54458e67-7d99-476e-930c-75848eb2c1b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03c88f56ad194520a936a77c7afde3e2', 'request_id': 'req-54458e67-7d99-476e-930c-75848eb2c1b4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:22.454546 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-54458e67-7d99-476e-930c-75848eb2c1b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03c88f56ad194520a936a77c7afde3e2', 'request_id': 'req-54458e67-7d99-476e-930c-75848eb2c1b4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:22.470679 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-54458e67-7d99-476e-930c-75848eb2c1b4 admin admin] The token cache is being invalidate because project 03c88f56ad194520a936a77c7afde3e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:22.470997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-54458e67-7d99-476e-930c-75848eb2c1b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:22.472231 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-54458e67-7d99-476e-930c-75848eb2c1b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:22.480081 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3864868-a0cc-4546-ab30-7d6f52527e32 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:22.480765 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1393/2769] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:22 2022] DELETE /identity/v3/projects/03c88f56ad194520a936a77c7afde3e2 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:22.481379 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3864868-a0cc-4546-ab30-7d6f52527e32 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:22.481778 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3864868-a0cc-4546-ab30-7d6f52527e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:22.504064 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54458e67-7d99-476e-930c-75848eb2c1b4 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:22.507944 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-54458e67-7d99-476e-930c-75848eb2c1b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:22.518448 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-d3864868-a0cc-4546-ab30-7d6f52527e32 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:22.519790 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d3864868-a0cc-4546-ab30-7d6f52527e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:22.611036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54458e67-7d99-476e-930c-75848eb2c1b4 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:22.617124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaffeb9-f407-4846-8604-39bb104e3e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:22.617386 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaffeb9-f407-4846-8604-39bb104e3e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:22.617629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaffeb9-f407-4846-8604-39bb104e3e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:22.617972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaffeb9-f407-4846-8604-39bb104e3e03 admin admin] PATH_INFO: `/v3/users/086e60f669894290be19bd9b4b40d1d5` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:22.619405 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaffeb9-f407-4846-8604-39bb104e3e03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=086e60f669894290be19bd9b4b40d1d5)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:22.622083 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaffeb9-f407-4846-8604-39bb104e3e03 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:22.642191 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:22.684624 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1377/2770] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:22.693878 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707edc3a-936d-4bea-890b-317f5293b48b None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:22.694205 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707edc3a-936d-4bea-890b-317f5293b48b None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:22.694556 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707edc3a-936d-4bea-890b-317f5293b48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:22.716227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7eaffeb9-f407-4846-8604-39bb104e3e03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '086e60f669894290be19bd9b4b40d1d5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:22.718810 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-707edc3a-936d-4bea-890b-317f5293b48b None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:22.720076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-707edc3a-936d-4bea-890b-317f5293b48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:22.730700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1378/2771] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:22.735472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707edc3a-936d-4bea-890b-317f5293b48b None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:22.742591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7eaffeb9-f407-4846-8604-39bb104e3e03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '086e60f669894290be19bd9b4b40d1d5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:22.746835 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707edc3a-936d-4bea-890b-317f5293b48b None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:22.754495 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78c556-0a21-4255-af06-3ae5f92a528f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:22.756408 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78c556-0a21-4255-af06-3ae5f92a528f admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:22.757343 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78c556-0a21-4255-af06-3ae5f92a528f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:22.758678 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78c556-0a21-4255-af06-3ae5f92a528f admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:22.759431 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78c556-0a21-4255-af06-3ae5f92a528f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:34:22.760002 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7eaffeb9-f407-4846-8604-39bb104e3e03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '086e60f669894290be19bd9b4b40d1d5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:22.760187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78c556-0a21-4255-af06-3ae5f92a528f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:22.766598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78c556-0a21-4255-af06-3ae5f92a528f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:22.767314 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7eaffeb9-f407-4846-8604-39bb104e3e03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '086e60f669894290be19bd9b4b40d1d5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:22.779429 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7eaffeb9-f407-4846-8604-39bb104e3e03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '086e60f669894290be19bd9b4b40d1d5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:22.784511 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1379/2772] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:34:22.789192 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1394/2773] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:22 2022] DELETE /identity/v3/users/086e60f669894290be19bd9b4b40d1d5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:34:22.790492 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78c556-0a21-4255-af06-3ae5f92a528f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:22.795560 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192432d-50ff-41b2-99dd-e3e24501d888 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:22.795843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192432d-50ff-41b2-99dd-e3e24501d888 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:22.796955 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192432d-50ff-41b2-99dd-e3e24501d888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:22.799028 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-cd78c556-0a21-4255-af06-3ae5f92a528f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:22.820996 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78c556-0a21-4255-af06-3ae5f92a528f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:22.830484 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9192432d-50ff-41b2-99dd-e3e24501d888 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:22.833272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9192432d-50ff-41b2-99dd-e3e24501d888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:22.859702 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:22.944847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86250674-50c5-4ecb-86f1-016dfb08fe5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:22.946014 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86250674-50c5-4ecb-86f1-016dfb08fe5f admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:22.946472 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86250674-50c5-4ecb-86f1-016dfb08fe5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:22.946921 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86250674-50c5-4ecb-86f1-016dfb08fe5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:22.947113 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1395/2774] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:22.948020 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86250674-50c5-4ecb-86f1-016dfb08fe5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:22.952474 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86250674-50c5-4ecb-86f1-016dfb08fe5f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:22.963943 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192432d-50ff-41b2-99dd-e3e24501d888 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:22.966687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9192432d-50ff-41b2-99dd-e3e24501d888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:23.005959 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1380/2775] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:22 2022] POST /identity/v3/projects => generated 383 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:23.008829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86250674-50c5-4ecb-86f1-016dfb08fe5f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:23.076793 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192432d-50ff-41b2-99dd-e3e24501d888 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:23.134911 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86250674-50c5-4ecb-86f1-016dfb08fe5f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:23.143742 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e49a50-580a-40bc-9d32-b5e3a1430b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:23.144703 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e49a50-580a-40bc-9d32-b5e3a1430b01 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.145213 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e49a50-580a-40bc-9d32-b5e3a1430b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.145685 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e49a50-580a-40bc-9d32-b5e3a1430b01 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.147056 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e49a50-580a-40bc-9d32-b5e3a1430b01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:23.151201 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e49a50-580a-40bc-9d32-b5e3a1430b01 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:23.188744 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87475a8-e869-402d-a60e-0a056a6aa40a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:23.190149 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87475a8-e869-402d-a60e-0a056a6aa40a service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.190373 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1381/2776] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:23 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:34:23.190580 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87475a8-e869-402d-a60e-0a056a6aa40a service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.190857 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87475a8-e869-402d-a60e-0a056a6aa40a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.197202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e49a50-580a-40bc-9d32-b5e3a1430b01 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:23.207773 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87475a8-e869-402d-a60e-0a056a6aa40a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:23.211236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87475a8-e869-402d-a60e-0a056a6aa40a service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:23.211876 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e49a50-580a-40bc-9d32-b5e3a1430b01 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:23.219549 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0143f-a49f-43c4-a582-73886ee1bda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:23.219549 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0143f-a49f-43c4-a582-73886ee1bda5 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.219549 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0143f-a49f-43c4-a582-73886ee1bda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.220224 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0143f-a49f-43c4-a582-73886ee1bda5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.221451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0143f-a49f-43c4-a582-73886ee1bda5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:23.223487 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0143f-a49f-43c4-a582-73886ee1bda5 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:23.230560 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1396/2777] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:23.247036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1382/2778] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:23.259646 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87475a8-e869-402d-a60e-0a056a6aa40a service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:23.292068 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87475a8-e869-402d-a60e-0a056a6aa40a service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:23.315225 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a4523-10a4-4e66-b4f2-4635bd182bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:23.316841 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a4523-10a4-4e66-b4f2-4635bd182bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.317411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a4523-10a4-4e66-b4f2-4635bd182bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.317878 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a4523-10a4-4e66-b4f2-4635bd182bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.318813 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a4523-10a4-4e66-b4f2-4635bd182bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:23.322329 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a4523-10a4-4e66-b4f2-4635bd182bb7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:23.343801 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1397/2779] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:23.349541 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0143f-a49f-43c4-a582-73886ee1bda5 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:23.363443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0143f-a49f-43c4-a582-73886ee1bda5 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:23.372318 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394faab-cbf4-42a1-9c82-2c63d0725a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:23.373565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394faab-cbf4-42a1-9c82-2c63d0725a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.373909 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394faab-cbf4-42a1-9c82-2c63d0725a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.374371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394faab-cbf4-42a1-9c82-2c63d0725a3f admin admin] PATH_INFO: `/v3/projects/030c47891dc8427792c6c7c3694b642d/users/6a94c77dd886493b8fd397e1f1c71073/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.398627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394faab-cbf4-42a1-9c82-2c63d0725a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=030c47891dc8427792c6c7c3694b642d, user_id=6a94c77dd886493b8fd397e1f1c71073, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:23.402502 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394faab-cbf4-42a1-9c82-2c63d0725a3f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:23.429131 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1383/2780] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:23 2022] PUT /identity/v3/projects/030c47891dc8427792c6c7c3694b642d/users/6a94c77dd886493b8fd397e1f1c71073/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:23.448083 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e7b95-8229-437a-8e35-2281866495ee None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.448603 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e7b95-8229-437a-8e35-2281866495ee None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.448955 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e7b95-8229-437a-8e35-2281866495ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.499975 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-3c0e7b95-8229-437a-8e35-2281866495ee None None] MFA Rules not processed for user `6a94c77dd886493b8fd397e1f1c71073`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:23.501886 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3c0e7b95-8229-437a-8e35-2281866495ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:23.576791 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394faab-cbf4-42a1-9c82-2c63d0725a3f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:23.579384 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6394faab-cbf4-42a1-9c82-2c63d0725a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:23.598479 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:23.609230 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:23.617406 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1398/2781] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:23.624034 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e7b95-8229-437a-8e35-2281866495ee None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:23.702423 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394faab-cbf4-42a1-9c82-2c63d0725a3f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:23.708320 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4ba56-8d59-413d-b157-97558e0c84c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:23.709182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4ba56-8d59-413d-b157-97558e0c84c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.709602 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4ba56-8d59-413d-b157-97558e0c84c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.710005 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4ba56-8d59-413d-b157-97558e0c84c8 admin admin] PATH_INFO: `/v3/projects/a07e102f326541e2a89cfe541aa7a022` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.711390 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4ba56-8d59-413d-b157-97558e0c84c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a07e102f326541e2a89cfe541aa7a022)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:23.713918 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4ba56-8d59-413d-b157-97558e0c84c8 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:23.738684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e7b95-8229-437a-8e35-2281866495ee None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:23.747561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd55a9-905f-4668-bf45-8702fa5828de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:23.750825 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd55a9-905f-4668-bf45-8702fa5828de admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.751900 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd55a9-905f-4668-bf45-8702fa5828de admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.752481 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd55a9-905f-4668-bf45-8702fa5828de admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.755140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd55a9-905f-4668-bf45-8702fa5828de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:23.758268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd55a9-905f-4668-bf45-8702fa5828de admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:23.813559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1399/2782] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:23 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:34:23.818811 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd55a9-905f-4668-bf45-8702fa5828de admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:23.819580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6ed4ba56-8d59-413d-b157-97558e0c84c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a07e102f326541e2a89cfe541aa7a022', 'request_id': 'req-6ed4ba56-8d59-413d-b157-97558e0c84c8'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:23.836063 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6ed4ba56-8d59-413d-b157-97558e0c84c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a07e102f326541e2a89cfe541aa7a022', 'request_id': 'req-6ed4ba56-8d59-413d-b157-97558e0c84c8'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:23.849458 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6ed4ba56-8d59-413d-b157-97558e0c84c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a07e102f326541e2a89cfe541aa7a022', 'request_id': 'req-6ed4ba56-8d59-413d-b157-97558e0c84c8'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:23.878686 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6ed4ba56-8d59-413d-b157-97558e0c84c8 admin admin] The token cache is being invalidate because project a07e102f326541e2a89cfe541aa7a022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:23.879551 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6ed4ba56-8d59-413d-b157-97558e0c84c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:23.880764 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6ed4ba56-8d59-413d-b157-97558e0c84c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:23.883660 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1384/2783] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:23 2022] DELETE /identity/v3/projects/a07e102f326541e2a89cfe541aa7a022 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:34:23.890576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f1693-c4d4-44a9-adee-373085312ae3 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.891089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f1693-c4d4-44a9-adee-373085312ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.891581 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f1693-c4d4-44a9-adee-373085312ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.927741 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-9c0f1693-c4d4-44a9-adee-373085312ae3 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:23.930103 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-9c0f1693-c4d4-44a9-adee-373085312ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:23.950580 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1385/2784] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:23.951984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd55a9-905f-4668-bf45-8702fa5828de admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:23.956957 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f7862-3d10-4cc4-87ba-a89729ba2742 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.957320 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f7862-3d10-4cc4-87ba-a89729ba2742 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.957786 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f7862-3d10-4cc4-87ba-a89729ba2742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.960895 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f3130-3697-46c3-8dc6-6b569f3a054c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:23.962021 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f3130-3697-46c3-8dc6-6b569f3a054c admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.963518 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f3130-3697-46c3-8dc6-6b569f3a054c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.964042 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f3130-3697-46c3-8dc6-6b569f3a054c admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.966196 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f3130-3697-46c3-8dc6-6b569f3a054c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:23.970124 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f3130-3697-46c3-8dc6-6b569f3a054c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:23.995585 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-d74f7862-3d10-4cc4-87ba-a89729ba2742 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:23.998397 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d74f7862-3d10-4cc4-87ba-a89729ba2742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:24.004073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1400/2785] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:23 2022] POST /identity/v3/users => generated 298 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:24.008139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f3130-3697-46c3-8dc6-6b569f3a054c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:24.010070 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b90f3130-3697-46c3-8dc6-6b569f3a054c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:24.013889 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1386/2786] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:24.021275 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f7862-3d10-4cc4-87ba-a89729ba2742 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:24.026785 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f3130-3697-46c3-8dc6-6b569f3a054c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:24.034310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57820199-e8e9-46ac-8ccf-0c3202e94b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:24.035126 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f7862-3d10-4cc4-87ba-a89729ba2742 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:24.035332 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57820199-e8e9-46ac-8ccf-0c3202e94b8f admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.035605 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57820199-e8e9-46ac-8ccf-0c3202e94b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.035896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57820199-e8e9-46ac-8ccf-0c3202e94b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.036589 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57820199-e8e9-46ac-8ccf-0c3202e94b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:24.038973 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57820199-e8e9-46ac-8ccf-0c3202e94b8f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:24.040643 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2a5e2-40f0-4adb-9493-069d884e0f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:24.041334 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2a5e2-40f0-4adb-9493-069d884e0f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.041612 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2a5e2-40f0-4adb-9493-069d884e0f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.041917 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2a5e2-40f0-4adb-9493-069d884e0f93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.042964 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2a5e2-40f0-4adb-9493-069d884e0f93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:34:24.043266 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2a5e2-40f0-4adb-9493-069d884e0f93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:24.045488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2a5e2-40f0-4adb-9493-069d884e0f93 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:24.060378 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1401/2787] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:34:24.065780 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57820199-e8e9-46ac-8ccf-0c3202e94b8f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:24.066412 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1387/2788] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:34:24.073363 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2a5e2-40f0-4adb-9493-069d884e0f93 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:24.085039 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57820199-e8e9-46ac-8ccf-0c3202e94b8f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:24.088311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2a5e2-40f0-4adb-9493-069d884e0f93 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:24.095105 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58445b-5aaf-4aab-bcc7-4c6efee9603f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:24.096268 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b98066-ee8f-47a8-ac24-08270b44ec21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:24.096427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58445b-5aaf-4aab-bcc7-4c6efee9603f admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.096912 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58445b-5aaf-4aab-bcc7-4c6efee9603f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.097428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58445b-5aaf-4aab-bcc7-4c6efee9603f admin admin] PATH_INFO: `/v3/projects/48e749726b5d4f9dbfbfa2224eefda29/users/bd0b06e5b9f84ccbb37df356851cd97b/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.097641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b98066-ee8f-47a8-ac24-08270b44ec21 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.097798 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b98066-ee8f-47a8-ac24-08270b44ec21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.098014 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b98066-ee8f-47a8-ac24-08270b44ec21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.098940 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b98066-ee8f-47a8-ac24-08270b44ec21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:24.101585 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b98066-ee8f-47a8-ac24-08270b44ec21 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:24.126912 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58445b-5aaf-4aab-bcc7-4c6efee9603f admin admin] RBAC: Authorizing `identity:create_grant(project_id=48e749726b5d4f9dbfbfa2224eefda29, user_id=bd0b06e5b9f84ccbb37df356851cd97b, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:24.132568 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58445b-5aaf-4aab-bcc7-4c6efee9603f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:24.146570 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1388/2789] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:24 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:34:24.152976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b98066-ee8f-47a8-ac24-08270b44ec21 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:24.159558 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1402/2790] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:24 2022] PUT /identity/v3/projects/48e749726b5d4f9dbfbfa2224eefda29/users/bd0b06e5b9f84ccbb37df356851cd97b/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:34:24.166053 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58445b-5aaf-4aab-bcc7-4c6efee9603f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:24.245395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:24.255553 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:24.274983 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b98066-ee8f-47a8-ac24-08270b44ec21 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:24.297465 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58445b-5aaf-4aab-bcc7-4c6efee9603f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:24.306567 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1ad32-a12e-4540-a761-46a1ea3b403b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:24.307637 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1ad32-a12e-4540-a761-46a1ea3b403b admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.308680 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1ad32-a12e-4540-a761-46a1ea3b403b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.309429 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1ad32-a12e-4540-a761-46a1ea3b403b admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.310632 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1ad32-a12e-4540-a761-46a1ea3b403b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:24.315633 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1ad32-a12e-4540-a761-46a1ea3b403b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:24.344605 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1403/2791] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:34:24.351930 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1ad32-a12e-4540-a761-46a1ea3b403b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:24.370265 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5e42a-fd74-4b53-bc94-cb0bd68bdb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:24.370923 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1ad32-a12e-4540-a761-46a1ea3b403b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:24.371598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5e42a-fd74-4b53-bc94-cb0bd68bdb50 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.372109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5e42a-fd74-4b53-bc94-cb0bd68bdb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.372611 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5e42a-fd74-4b53-bc94-cb0bd68bdb50 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.373814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5e42a-fd74-4b53-bc94-cb0bd68bdb50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:24.377875 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5e42a-fd74-4b53-bc94-cb0bd68bdb50 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:24.380221 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154e892-53d6-4e2c-ac4a-480f8f9d1d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:24.381451 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154e892-53d6-4e2c-ac4a-480f8f9d1d3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.381941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154e892-53d6-4e2c-ac4a-480f8f9d1d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.382527 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154e892-53d6-4e2c-ac4a-480f8f9d1d3f admin admin] PATH_INFO: `/v3/projects/48e749726b5d4f9dbfbfa2224eefda29/users/bd0b06e5b9f84ccbb37df356851cd97b/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.386261 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154e892-53d6-4e2c-ac4a-480f8f9d1d3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=48e749726b5d4f9dbfbfa2224eefda29, user_id=bd0b06e5b9f84ccbb37df356851cd97b, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:24.392217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154e892-53d6-4e2c-ac4a-480f8f9d1d3f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:24.414790 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1404/2792] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:24 2022] PUT /identity/v3/projects/48e749726b5d4f9dbfbfa2224eefda29/users/bd0b06e5b9f84ccbb37df356851cd97b/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:34:24.422435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1389/2793] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:24 2022] POST /identity/v3/users => generated 287 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:24.430032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154e892-53d6-4e2c-ac4a-480f8f9d1d3f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:24.434097 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb575c8-9f1e-42f6-9358-0e76d38256dc None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.434627 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb575c8-9f1e-42f6-9358-0e76d38256dc None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.435230 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb575c8-9f1e-42f6-9358-0e76d38256dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.491108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-3fb575c8-9f1e-42f6-9358-0e76d38256dc None None] MFA Rules not processed for user `bd0b06e5b9f84ccbb37df356851cd97b`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:24.493446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3fb575c8-9f1e-42f6-9358-0e76d38256dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:24.562304 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154e892-53d6-4e2c-ac4a-480f8f9d1d3f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:24.571022 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2c026-d8a1-4895-b564-292b23a6b7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:24.572036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2c026-d8a1-4895-b564-292b23a6b7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.572512 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2c026-d8a1-4895-b564-292b23a6b7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.572999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2c026-d8a1-4895-b564-292b23a6b7c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.574934 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2c026-d8a1-4895-b564-292b23a6b7c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:24.578494 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2c026-d8a1-4895-b564-292b23a6b7c4 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:24.604244 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1405/2794] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:24.605868 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1390/2795] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:24.607372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2c026-d8a1-4895-b564-292b23a6b7c4 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:24.625846 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb575c8-9f1e-42f6-9358-0e76d38256dc None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:24.627883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3fb575c8-9f1e-42f6-9358-0e76d38256dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:24.630342 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2c026-d8a1-4895-b564-292b23a6b7c4 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:24.642717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6326d-e167-4b74-9f38-0f8c050642d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:24.643762 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6326d-e167-4b74-9f38-0f8c050642d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.644421 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6326d-e167-4b74-9f38-0f8c050642d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.644889 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6326d-e167-4b74-9f38-0f8c050642d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.646131 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6326d-e167-4b74-9f38-0f8c050642d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:24.649339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6326d-e167-4b74-9f38-0f8c050642d3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:24.671109 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1406/2796] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:24.677242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6326d-e167-4b74-9f38-0f8c050642d3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:24.696223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6326d-e167-4b74-9f38-0f8c050642d3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:24.705907 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492c059-a316-47ef-959f-7dd4395afe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:24.706471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492c059-a316-47ef-959f-7dd4395afe6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.706962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492c059-a316-47ef-959f-7dd4395afe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.707455 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492c059-a316-47ef-959f-7dd4395afe6a admin admin] PATH_INFO: `/v3/projects/94a56fcfc827430fb1b73dcfd186024d/users/7a43d4ec9b1346e3a69b01d3d6e83a98/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.733197 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4492c059-a316-47ef-959f-7dd4395afe6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=94a56fcfc827430fb1b73dcfd186024d, user_id=7a43d4ec9b1346e3a69b01d3d6e83a98, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:24.736662 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4492c059-a316-47ef-959f-7dd4395afe6a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:24.737797 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb575c8-9f1e-42f6-9358-0e76d38256dc None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:24.748542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac941b-e592-4d1b-854b-cf1df8a89245 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:24.750441 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac941b-e592-4d1b-854b-cf1df8a89245 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.750877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac941b-e592-4d1b-854b-cf1df8a89245 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.751446 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac941b-e592-4d1b-854b-cf1df8a89245 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.769338 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1407/2797] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:24 2022] PUT /identity/v3/projects/94a56fcfc827430fb1b73dcfd186024d/users/7a43d4ec9b1346e3a69b01d3d6e83a98/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:34:24.771018 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070693e7-ddd1-4932-a7ba-282ca30f7040 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.771308 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070693e7-ddd1-4932-a7ba-282ca30f7040 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.771536 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac941b-e592-4d1b-854b-cf1df8a89245 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:24.771701 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070693e7-ddd1-4932-a7ba-282ca30f7040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.774270 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac941b-e592-4d1b-854b-cf1df8a89245 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:24.796050 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1391/2798] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:24 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:24.804686 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-070693e7-ddd1-4932-a7ba-282ca30f7040 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:24.807132 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-070693e7-ddd1-4932-a7ba-282ca30f7040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:24.912650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1408/2799] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:24.925771 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac941b-e592-4d1b-854b-cf1df8a89245 service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:24.927336 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0cac941b-e592-4d1b-854b-cf1df8a89245 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:24.948299 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:24.967629 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070693e7-ddd1-4932-a7ba-282ca30f7040 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:25.008978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:25.036697 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac941b-e592-4d1b-854b-cf1df8a89245 service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:25.073274 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070693e7-ddd1-4932-a7ba-282ca30f7040 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:25.081140 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07657316-cfa8-43f1-b89b-a2e65faf00f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:25.082110 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07657316-cfa8-43f1-b89b-a2e65faf00f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:25.082602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07657316-cfa8-43f1-b89b-a2e65faf00f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:25.083073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07657316-cfa8-43f1-b89b-a2e65faf00f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:25.087239 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-07657316-cfa8-43f1-b89b-a2e65faf00f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:25.132085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b1974-fcb2-4012-bcfa-3113a33cb5c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:25.132906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b1974-fcb2-4012-bcfa-3113a33cb5c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:25.133238 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b1974-fcb2-4012-bcfa-3113a33cb5c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:25.133509 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b1974-fcb2-4012-bcfa-3113a33cb5c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:25.146916 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b1974-fcb2-4012-bcfa-3113a33cb5c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:25.150120 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b1974-fcb2-4012-bcfa-3113a33cb5c4 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:25.167769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1392/2800] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:25.210830 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07657316-cfa8-43f1-b89b-a2e65faf00f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:25.214377 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07657316-cfa8-43f1-b89b-a2e65faf00f7 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:25.234804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1409/2801] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:28.368829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b1974-fcb2-4012-bcfa-3113a33cb5c4 service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:28.381850 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b1974-fcb2-4012-bcfa-3113a33cb5c4 service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:28.386927 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd21cf9-bd1e-43c1-a19e-e067e96425ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:28.387630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd21cf9-bd1e-43c1-a19e-e067e96425ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:28.387906 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd21cf9-bd1e-43c1-a19e-e067e96425ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:28.388178 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd21cf9-bd1e-43c1-a19e-e067e96425ee admin admin] PATH_INFO: `/v3/users/6a94c77dd886493b8fd397e1f1c71073` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:28.389401 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd21cf9-bd1e-43c1-a19e-e067e96425ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a94c77dd886493b8fd397e1f1c71073)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:28.391347 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd21cf9-bd1e-43c1-a19e-e067e96425ee admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:28.462505 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-3bd21cf9-bd1e-43c1-a19e-e067e96425ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a94c77dd886493b8fd397e1f1c71073'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:28.463794 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-3bd21cf9-bd1e-43c1-a19e-e067e96425ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a94c77dd886493b8fd397e1f1c71073'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:28.477260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-3bd21cf9-bd1e-43c1-a19e-e067e96425ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a94c77dd886493b8fd397e1f1c71073'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:28.504019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-3bd21cf9-bd1e-43c1-a19e-e067e96425ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a94c77dd886493b8fd397e1f1c71073'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:28.517829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-3bd21cf9-bd1e-43c1-a19e-e067e96425ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a94c77dd886493b8fd397e1f1c71073'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:28.535129 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1393/2802] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:28 2022] DELETE /identity/v3/users/6a94c77dd886493b8fd397e1f1c71073 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:28.542192 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373667e-8944-4177-a929-117a301b4613 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:28.543563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373667e-8944-4177-a929-117a301b4613 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:28.544219 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373667e-8944-4177-a929-117a301b4613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:28.590251 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c373667e-8944-4177-a929-117a301b4613 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:28.592217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c373667e-8944-4177-a929-117a301b4613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:28.736684 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1410/2803] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:28.751553 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd21cf9-bd1e-43c1-a19e-e067e96425ee admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:28.756079 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3bd21cf9-bd1e-43c1-a19e-e067e96425ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:28.876042 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:28.920403 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd21cf9-bd1e-43c1-a19e-e067e96425ee admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:28.926884 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e44fc-58f5-4d9c-a9b1-c88969584d5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:28.927717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e44fc-58f5-4d9c-a9b1-c88969584d5a service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:28.928153 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e44fc-58f5-4d9c-a9b1-c88969584d5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:28.928586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e44fc-58f5-4d9c-a9b1-c88969584d5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:28.951644 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e44fc-58f5-4d9c-a9b1-c88969584d5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:28.955973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e44fc-58f5-4d9c-a9b1-c88969584d5a service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:28.994213 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1394/2804] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:29.543917 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373667e-8944-4177-a929-117a301b4613 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:29.548585 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c373667e-8944-4177-a929-117a301b4613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:29.577561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373667e-8944-4177-a929-117a301b4613 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:29.587859 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67b592-abc1-4840-946a-ae54ef5ff580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:29.590982 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67b592-abc1-4840-946a-ae54ef5ff580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:29.592147 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67b592-abc1-4840-946a-ae54ef5ff580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:29.592968 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67b592-abc1-4840-946a-ae54ef5ff580 admin admin] PATH_INFO: `/v3/projects/030c47891dc8427792c6c7c3694b642d` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:29.596259 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67b592-abc1-4840-946a-ae54ef5ff580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=030c47891dc8427792c6c7c3694b642d)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:29.617263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67b592-abc1-4840-946a-ae54ef5ff580 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:29.768255 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-be67b592-abc1-4840-946a-ae54ef5ff580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '030c47891dc8427792c6c7c3694b642d', 'request_id': 'req-be67b592-abc1-4840-946a-ae54ef5ff580'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:29.788605 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-be67b592-abc1-4840-946a-ae54ef5ff580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '030c47891dc8427792c6c7c3694b642d', 'request_id': 'req-be67b592-abc1-4840-946a-ae54ef5ff580'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:29.813525 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-be67b592-abc1-4840-946a-ae54ef5ff580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '030c47891dc8427792c6c7c3694b642d', 'request_id': 'req-be67b592-abc1-4840-946a-ae54ef5ff580'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:29.841997 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-be67b592-abc1-4840-946a-ae54ef5ff580 admin admin] The token cache is being invalidate because project 030c47891dc8427792c6c7c3694b642d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:29.842469 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-be67b592-abc1-4840-946a-ae54ef5ff580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:29.843541 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-be67b592-abc1-4840-946a-ae54ef5ff580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:29.849575 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1411/2805] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:29 2022] DELETE /identity/v3/projects/030c47891dc8427792c6c7c3694b642d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:29.853832 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e44fc-58f5-4d9c-a9b1-c88969584d5a service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:29.875915 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e44fc-58f5-4d9c-a9b1-c88969584d5a service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:29.954089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046c6cd-d6b9-4b53-83e3-22e011ec609b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:29.955123 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046c6cd-d6b9-4b53-83e3-22e011ec609b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:29.955548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046c6cd-d6b9-4b53-83e3-22e011ec609b admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:29.955967 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046c6cd-d6b9-4b53-83e3-22e011ec609b admin admin] PATH_INFO: `/v3/users/bd0b06e5b9f84ccbb37df356851cd97b` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:29.957576 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046c6cd-d6b9-4b53-83e3-22e011ec609b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd0b06e5b9f84ccbb37df356851cd97b)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:29.960040 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046c6cd-d6b9-4b53-83e3-22e011ec609b admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:30.080381 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4046c6cd-d6b9-4b53-83e3-22e011ec609b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd0b06e5b9f84ccbb37df356851cd97b'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:30.082267 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4046c6cd-d6b9-4b53-83e3-22e011ec609b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd0b06e5b9f84ccbb37df356851cd97b'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:30.121448 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4046c6cd-d6b9-4b53-83e3-22e011ec609b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd0b06e5b9f84ccbb37df356851cd97b'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:30.152537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4046c6cd-d6b9-4b53-83e3-22e011ec609b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd0b06e5b9f84ccbb37df356851cd97b'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:30.169944 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-4046c6cd-d6b9-4b53-83e3-22e011ec609b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd0b06e5b9f84ccbb37df356851cd97b'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:30.184217 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1395/2806] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:29 2022] DELETE /identity/v3/users/bd0b06e5b9f84ccbb37df356851cd97b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:30.636741 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67b592-abc1-4840-946a-ae54ef5ff580 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:30.639454 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-be67b592-abc1-4840-946a-ae54ef5ff580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:30.661245 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:30.677039 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67b592-abc1-4840-946a-ae54ef5ff580 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:30.806561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ddb8e-9902-4ebb-bc64-28aff871b0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:30.807451 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ddb8e-9902-4ebb-bc64-28aff871b0c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:30.807834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ddb8e-9902-4ebb-bc64-28aff871b0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:30.809041 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ddb8e-9902-4ebb-bc64-28aff871b0c5 admin admin] PATH_INFO: `/v3/projects/48e749726b5d4f9dbfbfa2224eefda29` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:30.810747 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ddb8e-9902-4ebb-bc64-28aff871b0c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48e749726b5d4f9dbfbfa2224eefda29)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:30.813186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ddb8e-9902-4ebb-bc64-28aff871b0c5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:30.895815 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-fa4ddb8e-9902-4ebb-bc64-28aff871b0c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48e749726b5d4f9dbfbfa2224eefda29', 'request_id': 'req-fa4ddb8e-9902-4ebb-bc64-28aff871b0c5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:30.913862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-fa4ddb8e-9902-4ebb-bc64-28aff871b0c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48e749726b5d4f9dbfbfa2224eefda29', 'request_id': 'req-fa4ddb8e-9902-4ebb-bc64-28aff871b0c5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:30.932592 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-fa4ddb8e-9902-4ebb-bc64-28aff871b0c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48e749726b5d4f9dbfbfa2224eefda29', 'request_id': 'req-fa4ddb8e-9902-4ebb-bc64-28aff871b0c5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:30.962677 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-fa4ddb8e-9902-4ebb-bc64-28aff871b0c5 admin admin] The token cache is being invalidate because project 48e749726b5d4f9dbfbfa2224eefda29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:30.963097 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-fa4ddb8e-9902-4ebb-bc64-28aff871b0c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:30.964731 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-fa4ddb8e-9902-4ebb-bc64-28aff871b0c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:30.971428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1412/2807] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:30 2022] DELETE /identity/v3/projects/48e749726b5d4f9dbfbfa2224eefda29 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:30.973571 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e51004-f0c8-482f-a0af-061c3a798a0e None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:30.974291 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e51004-f0c8-482f-a0af-061c3a798a0e None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:30.975119 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e51004-f0c8-482f-a0af-061c3a798a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:31.028158 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-09e51004-f0c8-482f-a0af-061c3a798a0e None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:31.030630 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-09e51004-f0c8-482f-a0af-061c3a798a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:31.186802 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1396/2808] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:31.196335 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35647a04-0fac-4495-a923-a44ebe7236e8 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:31.196681 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35647a04-0fac-4495-a923-a44ebe7236e8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:31.196941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35647a04-0fac-4495-a923-a44ebe7236e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:31.238306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-35647a04-0fac-4495-a923-a44ebe7236e8 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:31.241826 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-35647a04-0fac-4495-a923-a44ebe7236e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:31.272833 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1413/2809] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:31.280327 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e51004-f0c8-482f-a0af-061c3a798a0e None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:31.297383 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:31.311976 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e51004-f0c8-482f-a0af-061c3a798a0e None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:31.323186 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedee90f-8546-4527-b2a1-7a302c40d9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:31.324757 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedee90f-8546-4527-b2a1-7a302c40d9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:31.325733 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedee90f-8546-4527-b2a1-7a302c40d9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:31.326184 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedee90f-8546-4527-b2a1-7a302c40d9b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:31.327324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedee90f-8546-4527-b2a1-7a302c40d9b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:34:31.327863 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedee90f-8546-4527-b2a1-7a302c40d9b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:31.332519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedee90f-8546-4527-b2a1-7a302c40d9b7 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:31.359100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1397/2810] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:34:31.365779 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35647a04-0fac-4495-a923-a44ebe7236e8 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:31.380270 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35647a04-0fac-4495-a923-a44ebe7236e8 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:31.399027 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada42d19-58ef-4ed4-97a8-d5308dc8b234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:31.400186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada42d19-58ef-4ed4-97a8-d5308dc8b234 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:31.400692 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada42d19-58ef-4ed4-97a8-d5308dc8b234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:31.401257 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada42d19-58ef-4ed4-97a8-d5308dc8b234 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:31.402543 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada42d19-58ef-4ed4-97a8-d5308dc8b234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:31.406881 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada42d19-58ef-4ed4-97a8-d5308dc8b234 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:31.469691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1414/2811] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:31 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:31.474539 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedee90f-8546-4527-b2a1-7a302c40d9b7 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:31.506639 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedee90f-8546-4527-b2a1-7a302c40d9b7 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:31.605652 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a9339-216b-4dd4-bd14-44522723cdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:31.606878 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a9339-216b-4dd4-bd14-44522723cdea admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:31.607325 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a9339-216b-4dd4-bd14-44522723cdea admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:31.607893 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a9339-216b-4dd4-bd14-44522723cdea admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:31.608970 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a9339-216b-4dd4-bd14-44522723cdea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:31.612864 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a9339-216b-4dd4-bd14-44522723cdea admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:31.665392 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1398/2812] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:31 2022] POST /identity/v3/users => generated 286 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:34:31.670913 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada42d19-58ef-4ed4-97a8-d5308dc8b234 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:31.747569 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada42d19-58ef-4ed4-97a8-d5308dc8b234 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:31.754903 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50885611-28e9-4438-83b8-9482ca5009fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:31.755981 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50885611-28e9-4438-83b8-9482ca5009fe admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:31.756195 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50885611-28e9-4438-83b8-9482ca5009fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:31.756388 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50885611-28e9-4438-83b8-9482ca5009fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:31.757187 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50885611-28e9-4438-83b8-9482ca5009fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:31.758942 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50885611-28e9-4438-83b8-9482ca5009fe admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:31.793139 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1415/2813] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:31.799421 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a9339-216b-4dd4-bd14-44522723cdea admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:31.819360 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a9339-216b-4dd4-bd14-44522723cdea admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:31.829211 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce51cd-dbb4-4f3f-bc1a-41532ebe0873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:31.830331 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce51cd-dbb4-4f3f-bc1a-41532ebe0873 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:31.830997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce51cd-dbb4-4f3f-bc1a-41532ebe0873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:31.831818 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce51cd-dbb4-4f3f-bc1a-41532ebe0873 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:31.833425 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce51cd-dbb4-4f3f-bc1a-41532ebe0873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:31.837248 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce51cd-dbb4-4f3f-bc1a-41532ebe0873 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:31.862426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1399/2814] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:31.868920 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50885611-28e9-4438-83b8-9482ca5009fe admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:31.895263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50885611-28e9-4438-83b8-9482ca5009fe admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:31.903601 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296431c-2077-4133-b56e-e53dd2c3c31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:31.904843 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296431c-2077-4133-b56e-e53dd2c3c31f admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:31.905460 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296431c-2077-4133-b56e-e53dd2c3c31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:31.906050 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296431c-2077-4133-b56e-e53dd2c3c31f admin admin] PATH_INFO: `/v3/projects/23dfe22266f14e1f95e291ed51e1eb81/users/614be48bcbaf4dd287b6e93d59a141d8/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:31.946026 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296431c-2077-4133-b56e-e53dd2c3c31f admin admin] RBAC: Authorizing `identity:create_grant(project_id=23dfe22266f14e1f95e291ed51e1eb81, user_id=614be48bcbaf4dd287b6e93d59a141d8, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:31.953411 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296431c-2077-4133-b56e-e53dd2c3c31f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:31.989275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1416/2815] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:31 2022] PUT /identity/v3/projects/23dfe22266f14e1f95e291ed51e1eb81/users/614be48bcbaf4dd287b6e93d59a141d8/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:34:31.990579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e791f-23b4-4877-b5de-f21bd46e3189 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:31.991373 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e791f-23b4-4877-b5de-f21bd46e3189 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:31.991524 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e791f-23b4-4877-b5de-f21bd46e3189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:32.038029 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-be7e791f-23b4-4877-b5de-f21bd46e3189 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:32.039829 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-be7e791f-23b4-4877-b5de-f21bd46e3189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:32.154236 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1400/2816] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:32.171673 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296431c-2077-4133-b56e-e53dd2c3c31f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:32.175073 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c296431c-2077-4133-b56e-e53dd2c3c31f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:32.258797 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:32.325858 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296431c-2077-4133-b56e-e53dd2c3c31f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:32.333781 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4d6c4-5a0a-4613-a923-799142d620a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:32.334459 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4d6c4-5a0a-4613-a923-799142d620a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:32.335021 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4d6c4-5a0a-4613-a923-799142d620a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:32.335595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4d6c4-5a0a-4613-a923-799142d620a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:32.357775 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4d6c4-5a0a-4613-a923-799142d620a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:32.361386 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4d6c4-5a0a-4613-a923-799142d620a4 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:32.394295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1417/2817] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:32.742067 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb99ca-1112-4729-84d3-acd00a4d2401 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:32.742577 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb99ca-1112-4729-84d3-acd00a4d2401 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:32.742834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb99ca-1112-4729-84d3-acd00a4d2401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:32.809897 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-b5eb99ca-1112-4729-84d3-acd00a4d2401 None None] MFA Rules not processed for user `7a43d4ec9b1346e3a69b01d3d6e83a98`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:32.814873 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-b5eb99ca-1112-4729-84d3-acd00a4d2401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:32.957849 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:32.965545 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1401/2818] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:32.969897 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4d6c4-5a0a-4613-a923-799142d620a4 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:32.973406 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8ad4d6c4-5a0a-4613-a923-799142d620a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:32.994255 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4d6c4-5a0a-4613-a923-799142d620a4 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:33.013799 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb11746-2e7e-41aa-b748-b4de6759e9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:33.016158 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb11746-2e7e-41aa-b748-b4de6759e9db admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:33.016994 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb11746-2e7e-41aa-b748-b4de6759e9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:33.017782 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb11746-2e7e-41aa-b748-b4de6759e9db admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:33.020448 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb11746-2e7e-41aa-b748-b4de6759e9db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:33.025301 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb11746-2e7e-41aa-b748-b4de6759e9db admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:33.084289 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1418/2819] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:32 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:33.091770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb99ca-1112-4729-84d3-acd00a4d2401 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:33.107185 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb99ca-1112-4729-84d3-acd00a4d2401 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:33.235647 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586fb8c3-fb93-4588-9b8a-c67cacbb47ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:33.236984 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fb8c3-fb93-4588-9b8a-c67cacbb47ae admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:33.237851 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fb8c3-fb93-4588-9b8a-c67cacbb47ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:33.238919 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fb8c3-fb93-4588-9b8a-c67cacbb47ae admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:33.239862 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586fb8c3-fb93-4588-9b8a-c67cacbb47ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:33.247323 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586fb8c3-fb93-4588-9b8a-c67cacbb47ae admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:33.290864 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1402/2820] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:33 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:34:33.296750 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb11746-2e7e-41aa-b748-b4de6759e9db admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:33.315804 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb11746-2e7e-41aa-b748-b4de6759e9db admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:33.324962 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589a4a1-894a-415b-9b26-3dbaf9eeba04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:33.326517 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589a4a1-894a-415b-9b26-3dbaf9eeba04 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:33.327102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589a4a1-894a-415b-9b26-3dbaf9eeba04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:33.327679 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589a4a1-894a-415b-9b26-3dbaf9eeba04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:33.330001 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589a4a1-894a-415b-9b26-3dbaf9eeba04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:33.332644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589a4a1-894a-415b-9b26-3dbaf9eeba04 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:33.357688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1419/2821] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:33.371235 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586fb8c3-fb93-4588-9b8a-c67cacbb47ae admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:33.391443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586fb8c3-fb93-4588-9b8a-c67cacbb47ae admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:33.403484 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95614204-303c-4612-8f43-b601dbda4c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:33.404422 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95614204-303c-4612-8f43-b601dbda4c46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:33.404883 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95614204-303c-4612-8f43-b601dbda4c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:33.405333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95614204-303c-4612-8f43-b601dbda4c46 admin admin] PATH_INFO: `/v3/projects/c582df208dec464aa84d4cf33190607a/users/91d9c01bf013482ba5cda775bb2972e6/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:33.443820 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95614204-303c-4612-8f43-b601dbda4c46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c582df208dec464aa84d4cf33190607a, user_id=91d9c01bf013482ba5cda775bb2972e6, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:33.450061 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95614204-303c-4612-8f43-b601dbda4c46 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:33.482752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1403/2822] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:33 2022] PUT /identity/v3/projects/c582df208dec464aa84d4cf33190607a/users/91d9c01bf013482ba5cda775bb2972e6/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:33.487279 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589a4a1-894a-415b-9b26-3dbaf9eeba04 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:33.502166 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589a4a1-894a-415b-9b26-3dbaf9eeba04 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:33.633614 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3649491-046a-4fff-a8f7-95a9928e2235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:33.634710 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3649491-046a-4fff-a8f7-95a9928e2235 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:33.636714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3649491-046a-4fff-a8f7-95a9928e2235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:33.637469 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3649491-046a-4fff-a8f7-95a9928e2235 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:33.638784 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3649491-046a-4fff-a8f7-95a9928e2235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:33.641905 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3649491-046a-4fff-a8f7-95a9928e2235 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:33.671521 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1420/2823] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:33.673457 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95614204-303c-4612-8f43-b601dbda4c46 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:33.694463 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95614204-303c-4612-8f43-b601dbda4c46 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:33.706895 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb036b-c851-4010-899e-0385b7ff1ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:33.707216 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb036b-c851-4010-899e-0385b7ff1ce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:33.707554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb036b-c851-4010-899e-0385b7ff1ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:33.708051 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb036b-c851-4010-899e-0385b7ff1ce2 admin admin] PATH_INFO: `/v3/projects/c582df208dec464aa84d4cf33190607a/users/91d9c01bf013482ba5cda775bb2972e6/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:33.712514 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb036b-c851-4010-899e-0385b7ff1ce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c582df208dec464aa84d4cf33190607a, user_id=91d9c01bf013482ba5cda775bb2972e6, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:33.718953 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb036b-c851-4010-899e-0385b7ff1ce2 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:33.744311 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1404/2824] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:33 2022] PUT /identity/v3/projects/c582df208dec464aa84d4cf33190607a/users/91d9c01bf013482ba5cda775bb2972e6/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:33.799345 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c71218-f332-412e-8281-9bc4093c0a08 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:33.799871 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c71218-f332-412e-8281-9bc4093c0a08 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:33.800385 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c71218-f332-412e-8281-9bc4093c0a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:33.866759 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c2c71218-f332-412e-8281-9bc4093c0a08 None None] MFA Rules not processed for user `614be48bcbaf4dd287b6e93d59a141d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:33.869217 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c2c71218-f332-412e-8281-9bc4093c0a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:34.047602 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1421/2825] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:34.054601 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb036b-c851-4010-899e-0385b7ff1ce2 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:34.203434 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb036b-c851-4010-899e-0385b7ff1ce2 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:34.217804 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1d48a-7e93-4395-a815-74cd918d2733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:34.218841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1d48a-7e93-4395-a815-74cd918d2733 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:34.219411 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1d48a-7e93-4395-a815-74cd918d2733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:34.220080 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1d48a-7e93-4395-a815-74cd918d2733 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:34.221582 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1d48a-7e93-4395-a815-74cd918d2733 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:34.225756 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1d48a-7e93-4395-a815-74cd918d2733 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:34.284380 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1405/2826] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:34 2022] POST /identity/v3/projects => generated 373 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:34.289800 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c71218-f332-412e-8281-9bc4093c0a08 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:34.307687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c71218-f332-412e-8281-9bc4093c0a08 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:34.351979 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:34.425312 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacd70e-a297-4960-ab98-88c58cdc33d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:34.426610 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacd70e-a297-4960-ab98-88c58cdc33d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:34.427178 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacd70e-a297-4960-ab98-88c58cdc33d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:34.427952 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacd70e-a297-4960-ab98-88c58cdc33d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:34.429264 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deacd70e-a297-4960-ab98-88c58cdc33d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:34.434268 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deacd70e-a297-4960-ab98-88c58cdc33d8 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:34.490190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1422/2827] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:34 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:34.492121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1d48a-7e93-4395-a815-74cd918d2733 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:34.514713 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1d48a-7e93-4395-a815-74cd918d2733 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:34.533797 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acabe1-c73b-4c96-9ee1-2831935758f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:34.534841 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acabe1-c73b-4c96-9ee1-2831935758f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:34.535198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acabe1-c73b-4c96-9ee1-2831935758f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:34.535438 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acabe1-c73b-4c96-9ee1-2831935758f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:34.540089 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acabe1-c73b-4c96-9ee1-2831935758f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:34.544287 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acabe1-c73b-4c96-9ee1-2831935758f6 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:34.579200 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1406/2828] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:34.586165 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacd70e-a297-4960-ab98-88c58cdc33d8 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:34.624573 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacd70e-a297-4960-ab98-88c58cdc33d8 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:34.637372 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2590e0-4bbd-4a19-86c3-1e2ff7ed06ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:34.646210 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2590e0-4bbd-4a19-86c3-1e2ff7ed06ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:34.648136 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2590e0-4bbd-4a19-86c3-1e2ff7ed06ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:34.648776 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2590e0-4bbd-4a19-86c3-1e2ff7ed06ee admin admin] PATH_INFO: `/v3/projects/ecbb2e2a7838483ba559637b0e7a9b91/users/7b594556bc944e1fb429d5ed7a5dd0a3/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:34.686956 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2590e0-4bbd-4a19-86c3-1e2ff7ed06ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecbb2e2a7838483ba559637b0e7a9b91, user_id=7b594556bc944e1fb429d5ed7a5dd0a3, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:34.692650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2590e0-4bbd-4a19-86c3-1e2ff7ed06ee admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:34.722552 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1423/2829] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:34 2022] PUT /identity/v3/projects/ecbb2e2a7838483ba559637b0e7a9b91/users/7b594556bc944e1fb429d5ed7a5dd0a3/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:34.732218 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acabe1-c73b-4c96-9ee1-2831935758f6 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:34.755479 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acabe1-c73b-4c96-9ee1-2831935758f6 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:34.815458 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:34.896470 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101bf1d-170a-42a7-81a9-a131b44b6a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:34.898044 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101bf1d-170a-42a7-81a9-a131b44b6a4f admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:34.898599 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101bf1d-170a-42a7-81a9-a131b44b6a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:34.899197 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101bf1d-170a-42a7-81a9-a131b44b6a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:34.900651 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101bf1d-170a-42a7-81a9-a131b44b6a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:34.907595 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101bf1d-170a-42a7-81a9-a131b44b6a4f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:34.940603 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1407/2830] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:34.943820 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2590e0-4bbd-4a19-86c3-1e2ff7ed06ee admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:34.980156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2590e0-4bbd-4a19-86c3-1e2ff7ed06ee admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:34.988420 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039af9af-6418-4106-b05f-79856f00dd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:34.989714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039af9af-6418-4106-b05f-79856f00dd59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:34.990216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039af9af-6418-4106-b05f-79856f00dd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:34.990690 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039af9af-6418-4106-b05f-79856f00dd59 admin admin] PATH_INFO: `/v3/projects/ecbb2e2a7838483ba559637b0e7a9b91/users/7b594556bc944e1fb429d5ed7a5dd0a3/roles/dc5d71ab685e420697f520aa3a368177` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:35.000535 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039af9af-6418-4106-b05f-79856f00dd59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecbb2e2a7838483ba559637b0e7a9b91, user_id=7b594556bc944e1fb429d5ed7a5dd0a3, role_id=dc5d71ab685e420697f520aa3a368177)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:35.010295 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039af9af-6418-4106-b05f-79856f00dd59 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:35.035181 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1424/2831] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:34 2022] PUT /identity/v3/projects/ecbb2e2a7838483ba559637b0e7a9b91/users/7b594556bc944e1fb429d5ed7a5dd0a3/roles/dc5d71ab685e420697f520aa3a368177 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:36.875183 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8744e4-d8ed-4447-9cac-6dacf4665669 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:36.876093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8744e4-d8ed-4447-9cac-6dacf4665669 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:36.876093 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8744e4-d8ed-4447-9cac-6dacf4665669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:36.935479 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-3c8744e4-d8ed-4447-9cac-6dacf4665669 None None] MFA Rules not processed for user `7b594556bc944e1fb429d5ed7a5dd0a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:36.937744 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3c8744e4-d8ed-4447-9cac-6dacf4665669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:37.093163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1408/2832] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:37.110704 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039af9af-6418-4106-b05f-79856f00dd59 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:37.113297 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-039af9af-6418-4106-b05f-79856f00dd59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:37.223980 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039af9af-6418-4106-b05f-79856f00dd59 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:37.228971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae8c1a-bc68-4131-ac49-26d7d8975fc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:37.229860 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae8c1a-bc68-4131-ac49-26d7d8975fc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:37.230148 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae8c1a-bc68-4131-ac49-26d7d8975fc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:37.230489 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae8c1a-bc68-4131-ac49-26d7d8975fc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:37.353416 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae8c1a-bc68-4131-ac49-26d7d8975fc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:37.356675 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae8c1a-bc68-4131-ac49-26d7d8975fc5 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:37.381464 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1425/2833] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:39.248836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8744e4-d8ed-4447-9cac-6dacf4665669 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:39.251395 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3c8744e4-d8ed-4447-9cac-6dacf4665669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:39.272542 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8744e4-d8ed-4447-9cac-6dacf4665669 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:39.290593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55006a37-0036-4e1c-aed3-83cfd8eedda3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:39.292127 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55006a37-0036-4e1c-aed3-83cfd8eedda3 service cinder] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:39.292879 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55006a37-0036-4e1c-aed3-83cfd8eedda3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:39.293624 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55006a37-0036-4e1c-aed3-83cfd8eedda3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:39.320096 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55006a37-0036-4e1c-aed3-83cfd8eedda3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:39.323670 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55006a37-0036-4e1c-aed3-83cfd8eedda3 service cinder] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:39.349532 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1409/2834] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:42.126465 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2241e7d-e376-42bd-bafc-b561ea4fd292 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:42.127301 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2241e7d-e376-42bd-bafc-b561ea4fd292 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:42.127478 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2241e7d-e376-42bd-bafc-b561ea4fd292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:42.178076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f2241e7d-e376-42bd-bafc-b561ea4fd292 None None] MFA Rules not processed for user `91d9c01bf013482ba5cda775bb2972e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:42.180381 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f2241e7d-e376-42bd-bafc-b561ea4fd292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:42.218316 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:42.299872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1426/2835] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:42.317747 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55006a37-0036-4e1c-aed3-83cfd8eedda3 service cinder] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:42.319297 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-55006a37-0036-4e1c-aed3-83cfd8eedda3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:42.418696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55006a37-0036-4e1c-aed3-83cfd8eedda3 service cinder] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:42.425433 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68172e-56a2-4b3c-9f16-40080a7f3748 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:42.426072 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68172e-56a2-4b3c-9f16-40080a7f3748 service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:42.426364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68172e-56a2-4b3c-9f16-40080a7f3748 service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:42.426669 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68172e-56a2-4b3c-9f16-40080a7f3748 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:42.480746 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=100 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:42.535324 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68172e-56a2-4b3c-9f16-40080a7f3748 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:42.538565 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68172e-56a2-4b3c-9f16-40080a7f3748 service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:42.558729 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1410/2836] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:44.522114 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098ebe0-24aa-4d00-bb71-0dabfab34393 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:44.523579 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098ebe0-24aa-4d00-bb71-0dabfab34393 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:44.524371 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098ebe0-24aa-4d00-bb71-0dabfab34393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:44.572299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0098ebe0-24aa-4d00-bb71-0dabfab34393 None None] MFA Rules not processed for user `7a43d4ec9b1346e3a69b01d3d6e83a98`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:44.575711 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0098ebe0-24aa-4d00-bb71-0dabfab34393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:44.596548 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1427/2837] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:52.550103 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=13 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:54.564496 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=23 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:55.248700 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68172e-56a2-4b3c-9f16-40080a7f3748 service nova] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:55.356303 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68172e-56a2-4b3c-9f16-40080a7f3748 service nova] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:55.361831 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf40a9-0b3a-46f9-995e-19a5fd4dd144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:55.362508 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf40a9-0b3a-46f9-995e-19a5fd4dd144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:55.362811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf40a9-0b3a-46f9-995e-19a5fd4dd144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:55.363102 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf40a9-0b3a-46f9-995e-19a5fd4dd144 admin admin] PATH_INFO: `/v3/users/614be48bcbaf4dd287b6e93d59a141d8` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:55.364375 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf40a9-0b3a-46f9-995e-19a5fd4dd144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=614be48bcbaf4dd287b6e93d59a141d8)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:55.366387 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf40a9-0b3a-46f9-995e-19a5fd4dd144 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:55.442237 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6adf40a9-0b3a-46f9-995e-19a5fd4dd144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '614be48bcbaf4dd287b6e93d59a141d8'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:55.443316 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6adf40a9-0b3a-46f9-995e-19a5fd4dd144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '614be48bcbaf4dd287b6e93d59a141d8'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:55.455693 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6adf40a9-0b3a-46f9-995e-19a5fd4dd144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '614be48bcbaf4dd287b6e93d59a141d8'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:55.475569 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6adf40a9-0b3a-46f9-995e-19a5fd4dd144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '614be48bcbaf4dd287b6e93d59a141d8'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:55.487519 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-6adf40a9-0b3a-46f9-995e-19a5fd4dd144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '614be48bcbaf4dd287b6e93d59a141d8'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:55.497425 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1411/2838] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:55 2022] DELETE /identity/v3/users/614be48bcbaf4dd287b6e93d59a141d8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:55.719123 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098ebe0-24aa-4d00-bb71-0dabfab34393 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:55.738272 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098ebe0-24aa-4d00-bb71-0dabfab34393 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:55.831475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7b71b-2042-42f1-aebc-e7e93583edbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:55.831989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7b71b-2042-42f1-aebc-e7e93583edbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:55.832545 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7b71b-2042-42f1-aebc-e7e93583edbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:55.832828 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7b71b-2042-42f1-aebc-e7e93583edbb admin admin] PATH_INFO: `/v3/projects/23dfe22266f14e1f95e291ed51e1eb81` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:55.834339 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7b71b-2042-42f1-aebc-e7e93583edbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=23dfe22266f14e1f95e291ed51e1eb81)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:55.836823 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7b71b-2042-42f1-aebc-e7e93583edbb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:55.908204 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-efa7b71b-2042-42f1-aebc-e7e93583edbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23dfe22266f14e1f95e291ed51e1eb81', 'request_id': 'req-efa7b71b-2042-42f1-aebc-e7e93583edbb'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:55.915873 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-efa7b71b-2042-42f1-aebc-e7e93583edbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23dfe22266f14e1f95e291ed51e1eb81', 'request_id': 'req-efa7b71b-2042-42f1-aebc-e7e93583edbb'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:55.928185 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-efa7b71b-2042-42f1-aebc-e7e93583edbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23dfe22266f14e1f95e291ed51e1eb81', 'request_id': 'req-efa7b71b-2042-42f1-aebc-e7e93583edbb'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:55.945453 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-efa7b71b-2042-42f1-aebc-e7e93583edbb admin admin] The token cache is being invalidate because project 23dfe22266f14e1f95e291ed51e1eb81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:55.945883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-efa7b71b-2042-42f1-aebc-e7e93583edbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:55.946653 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-efa7b71b-2042-42f1-aebc-e7e93583edbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:55.948335 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1428/2839] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:55 2022] DELETE /identity/v3/projects/23dfe22266f14e1f95e291ed51e1eb81 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:55.952997 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf40a9-0b3a-46f9-995e-19a5fd4dd144 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:55.954868 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-6adf40a9-0b3a-46f9-995e-19a5fd4dd144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:55.974792 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf40a9-0b3a-46f9-995e-19a5fd4dd144 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:56.063504 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbffb9-2508-417e-a89b-873d95b04f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:56.064170 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbffb9-2508-417e-a89b-873d95b04f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:56.064451 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbffb9-2508-417e-a89b-873d95b04f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:56.064721 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbffb9-2508-417e-a89b-873d95b04f11 admin admin] PATH_INFO: `/v3/users/7b594556bc944e1fb429d5ed7a5dd0a3` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:56.070243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbffb9-2508-417e-a89b-873d95b04f11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b594556bc944e1fb429d5ed7a5dd0a3)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:56.070243 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbffb9-2508-417e-a89b-873d95b04f11 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:56.107465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:56.148466 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-75dbffb9-2508-417e-a89b-873d95b04f11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b594556bc944e1fb429d5ed7a5dd0a3'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:56.149513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-75dbffb9-2508-417e-a89b-873d95b04f11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b594556bc944e1fb429d5ed7a5dd0a3'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:56.163535 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-75dbffb9-2508-417e-a89b-873d95b04f11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b594556bc944e1fb429d5ed7a5dd0a3'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:56.181250 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-75dbffb9-2508-417e-a89b-873d95b04f11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b594556bc944e1fb429d5ed7a5dd0a3'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:56.191298 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-75dbffb9-2508-417e-a89b-873d95b04f11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b594556bc944e1fb429d5ed7a5dd0a3'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:56.199711 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1412/2840] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:55 2022] DELETE /identity/v3/users/7b594556bc944e1fb429d5ed7a5dd0a3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:56.447454 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7b71b-2042-42f1-aebc-e7e93583edbb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:56.450820 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-efa7b71b-2042-42f1-aebc-e7e93583edbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:56.471011 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7b71b-2042-42f1-aebc-e7e93583edbb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:56.590475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60898b6c-d7c9-4266-88d5-4abda0eb4763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:56.591599 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60898b6c-d7c9-4266-88d5-4abda0eb4763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:56.592112 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60898b6c-d7c9-4266-88d5-4abda0eb4763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:56.592619 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60898b6c-d7c9-4266-88d5-4abda0eb4763 admin admin] PATH_INFO: `/v3/projects/ecbb2e2a7838483ba559637b0e7a9b91` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:56.594859 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60898b6c-d7c9-4266-88d5-4abda0eb4763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecbb2e2a7838483ba559637b0e7a9b91)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:56.598176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60898b6c-d7c9-4266-88d5-4abda0eb4763 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:56.659941 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:56.681216 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-60898b6c-d7c9-4266-88d5-4abda0eb4763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecbb2e2a7838483ba559637b0e7a9b91', 'request_id': 'req-60898b6c-d7c9-4266-88d5-4abda0eb4763'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:56.698901 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-60898b6c-d7c9-4266-88d5-4abda0eb4763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecbb2e2a7838483ba559637b0e7a9b91', 'request_id': 'req-60898b6c-d7c9-4266-88d5-4abda0eb4763'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:56.711180 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-60898b6c-d7c9-4266-88d5-4abda0eb4763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecbb2e2a7838483ba559637b0e7a9b91', 'request_id': 'req-60898b6c-d7c9-4266-88d5-4abda0eb4763'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:56.727153 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-60898b6c-d7c9-4266-88d5-4abda0eb4763 admin admin] The token cache is being invalidate because project ecbb2e2a7838483ba559637b0e7a9b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:56.728283 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-60898b6c-d7c9-4266-88d5-4abda0eb4763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:56.728471 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-60898b6c-d7c9-4266-88d5-4abda0eb4763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:56.731121 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1429/2841] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:56 2022] DELETE /identity/v3/projects/ecbb2e2a7838483ba559637b0e7a9b91 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:56.736347 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816b2bc-5833-42de-98e8-c35c84e7ac69 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:56.736962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816b2bc-5833-42de-98e8-c35c84e7ac69 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:56.736962 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816b2bc-5833-42de-98e8-c35c84e7ac69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:56.761352 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-b816b2bc-5833-42de-98e8-c35c84e7ac69 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:56.763386 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-b816b2bc-5833-42de-98e8-c35c84e7ac69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:56.842776 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1413/2842] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:56.854630 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2ca45-501d-47e2-8afd-3602c52e3533 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:56.855084 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2ca45-501d-47e2-8afd-3602c52e3533 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:56.855564 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2ca45-501d-47e2-8afd-3602c52e3533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:56.880231 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-83b2ca45-501d-47e2-8afd-3602c52e3533 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:56.882524 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-83b2ca45-501d-47e2-8afd-3602c52e3533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:56.897561 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1430/2843] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:56.904772 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816b2bc-5833-42de-98e8-c35c84e7ac69 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:56.923317 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816b2bc-5833-42de-98e8-c35c84e7ac69 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:56.931707 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95c372-0525-405c-98ac-04370e645e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:56.933408 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95c372-0525-405c-98ac-04370e645e16 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:56.933905 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95c372-0525-405c-98ac-04370e645e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:56.934396 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95c372-0525-405c-98ac-04370e645e16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:56.935619 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d95c372-0525-405c-98ac-04370e645e16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81679) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:34:56.936189 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d95c372-0525-405c-98ac-04370e645e16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:56.939345 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d95c372-0525-405c-98ac-04370e645e16 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:56.966743 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1414/2844] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:34:56.972849 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2ca45-501d-47e2-8afd-3602c52e3533 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:56.985359 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2ca45-501d-47e2-8afd-3602c52e3533 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:56.991990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a82da6-c960-4862-a212-5bb6d47d6ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:56.992777 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a82da6-c960-4862-a212-5bb6d47d6ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:56.993085 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a82da6-c960-4862-a212-5bb6d47d6ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:56.993425 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a82da6-c960-4862-a212-5bb6d47d6ca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:56.994152 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a82da6-c960-4862-a212-5bb6d47d6ca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:56.998115 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a82da6-c960-4862-a212-5bb6d47d6ca2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:57.036044 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1431/2845] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:56 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:34:57.042252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95c372-0525-405c-98ac-04370e645e16 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:57.056650 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95c372-0525-405c-98ac-04370e645e16 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:57.131498 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bb7b7-1744-46a1-8b0f-4cbe3519a0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:57.132143 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bb7b7-1744-46a1-8b0f-4cbe3519a0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:57.132430 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bb7b7-1744-46a1-8b0f-4cbe3519a0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:57.132710 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bb7b7-1744-46a1-8b0f-4cbe3519a0f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:57.133440 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869bb7b7-1744-46a1-8b0f-4cbe3519a0f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:57.136297 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869bb7b7-1744-46a1-8b0f-4cbe3519a0f8 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:57.167058 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1415/2846] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:57 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:57.171954 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a82da6-c960-4862-a212-5bb6d47d6ca2 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:57.185430 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a82da6-c960-4862-a212-5bb6d47d6ca2 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:57.191009 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1648b1f-8684-4782-9662-1569ac2cc2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:57.191694 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1648b1f-8684-4782-9662-1569ac2cc2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:57.191951 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1648b1f-8684-4782-9662-1569ac2cc2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:57.192306 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1648b1f-8684-4782-9662-1569ac2cc2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:57.193028 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1648b1f-8684-4782-9662-1569ac2cc2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:57.195190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1648b1f-8684-4782-9662-1569ac2cc2d4 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:57.209896 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1432/2847] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:57.216340 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bb7b7-1744-46a1-8b0f-4cbe3519a0f8 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:57.228586 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bb7b7-1744-46a1-8b0f-4cbe3519a0f8 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:57.235763 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a6bf1-e034-4abd-ae91-d2a7c324d3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:57.236729 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a6bf1-e034-4abd-ae91-d2a7c324d3eb admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:57.237271 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a6bf1-e034-4abd-ae91-d2a7c324d3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:57.237679 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a6bf1-e034-4abd-ae91-d2a7c324d3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:57.238513 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a6bf1-e034-4abd-ae91-d2a7c324d3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:57.240242 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a6bf1-e034-4abd-ae91-d2a7c324d3eb admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:57.256636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1416/2848] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:57.265319 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1648b1f-8684-4782-9662-1569ac2cc2d4 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:57.285014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1648b1f-8684-4782-9662-1569ac2cc2d4 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:57.294171 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07201f5b-5444-4b72-844d-dfbfd73cf2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:57.295420 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07201f5b-5444-4b72-844d-dfbfd73cf2ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:57.295930 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07201f5b-5444-4b72-844d-dfbfd73cf2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:57.296448 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07201f5b-5444-4b72-844d-dfbfd73cf2ec admin admin] PATH_INFO: `/v3/projects/c3b50c56130b488e9d1d44e6af7b4ee1/users/5df69bf067bc4b2fa5c8041409f919e0/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:57.333894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07201f5b-5444-4b72-844d-dfbfd73cf2ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3b50c56130b488e9d1d44e6af7b4ee1, user_id=5df69bf067bc4b2fa5c8041409f919e0, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:57.339299 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07201f5b-5444-4b72-844d-dfbfd73cf2ec admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:57.365543 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1433/2849] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:57 2022] PUT /identity/v3/projects/c3b50c56130b488e9d1d44e6af7b4ee1/users/5df69bf067bc4b2fa5c8041409f919e0/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:57.379198 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837f731-4310-40c6-b01c-ed7379b56816 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:57.379798 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837f731-4310-40c6-b01c-ed7379b56816 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:57.380280 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837f731-4310-40c6-b01c-ed7379b56816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:57.394328 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:57.436330 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-a837f731-4310-40c6-b01c-ed7379b56816 None None] MFA Rules not processed for user `5df69bf067bc4b2fa5c8041409f919e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:57.437697 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-a837f731-4310-40c6-b01c-ed7379b56816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:57.519938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1417/2850] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:34:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:57.532105 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07201f5b-5444-4b72-844d-dfbfd73cf2ec admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:57.533913 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-07201f5b-5444-4b72-844d-dfbfd73cf2ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:57.603102 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07201f5b-5444-4b72-844d-dfbfd73cf2ec admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:57.607539 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f2f41-8c33-4dd5-b401-9145afd102f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:57.608211 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f2f41-8c33-4dd5-b401-9145afd102f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:57.608500 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f2f41-8c33-4dd5-b401-9145afd102f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:57.608771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f2f41-8c33-4dd5-b401-9145afd102f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:57.619573 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f2f41-8c33-4dd5-b401-9145afd102f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:57.621186 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f2f41-8c33-4dd5-b401-9145afd102f3 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:57.632902 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1434/2851] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:35:00.911002 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837f731-4310-40c6-b01c-ed7379b56816 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:01.002938 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837f731-4310-40c6-b01c-ed7379b56816 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:01.010055 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93891b78-1dcf-468b-9cc7-f56cef5c2bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:01.011164 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93891b78-1dcf-468b-9cc7-f56cef5c2bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:01.011640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93891b78-1dcf-468b-9cc7-f56cef5c2bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:01.012149 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93891b78-1dcf-468b-9cc7-f56cef5c2bd3 admin admin] PATH_INFO: `/v3/users/5df69bf067bc4b2fa5c8041409f919e0` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:01.014184 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93891b78-1dcf-468b-9cc7-f56cef5c2bd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5df69bf067bc4b2fa5c8041409f919e0)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:01.017986 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93891b78-1dcf-468b-9cc7-f56cef5c2bd3 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:01.091523 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-93891b78-1dcf-468b-9cc7-f56cef5c2bd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5df69bf067bc4b2fa5c8041409f919e0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:01.092161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-93891b78-1dcf-468b-9cc7-f56cef5c2bd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5df69bf067bc4b2fa5c8041409f919e0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:01.100641 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-93891b78-1dcf-468b-9cc7-f56cef5c2bd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5df69bf067bc4b2fa5c8041409f919e0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:01.116507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-93891b78-1dcf-468b-9cc7-f56cef5c2bd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5df69bf067bc4b2fa5c8041409f919e0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:01.127922 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-93891b78-1dcf-468b-9cc7-f56cef5c2bd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5df69bf067bc4b2fa5c8041409f919e0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:01.139690 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1418/2852] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:35:00 2022] DELETE /identity/v3/users/5df69bf067bc4b2fa5c8041409f919e0 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:35:01.145936 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da0978-73ef-4d6a-9be3-82a055a9e8c6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:01.146880 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da0978-73ef-4d6a-9be3-82a055a9e8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:01.146880 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da0978-73ef-4d6a-9be3-82a055a9e8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:01.179990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b0da0978-73ef-4d6a-9be3-82a055a9e8c6 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:35:01.182263 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b0da0978-73ef-4d6a-9be3-82a055a9e8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:01.286309 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:35:01.299032 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1435/2853] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:35:01.311514 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93891b78-1dcf-468b-9cc7-f56cef5c2bd3 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:01.313585 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-93891b78-1dcf-468b-9cc7-f56cef5c2bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:01.331590 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:35:01.398181 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93891b78-1dcf-468b-9cc7-f56cef5c2bd3 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:01.410568 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba4ed0-9307-48a0-9bde-913f90a49c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:01.411246 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba4ed0-9307-48a0-9bde-913f90a49c5e service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:01.411537 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba4ed0-9307-48a0-9bde-913f90a49c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:01.411818 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba4ed0-9307-48a0-9bde-913f90a49c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:01.426968 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba4ed0-9307-48a0-9bde-913f90a49c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:01.429447 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba4ed0-9307-48a0-9bde-913f90a49c5e service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:01.448180 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1419/2854] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:35:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:35:01.845259 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da0978-73ef-4d6a-9be3-82a055a9e8c6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:01.847869 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b0da0978-73ef-4d6a-9be3-82a055a9e8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:01.959989 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da0978-73ef-4d6a-9be3-82a055a9e8c6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:01.967082 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa66855-b6d1-4919-8422-65488ad85e63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:01.967872 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa66855-b6d1-4919-8422-65488ad85e63 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:01.968287 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa66855-b6d1-4919-8422-65488ad85e63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:01.968809 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa66855-b6d1-4919-8422-65488ad85e63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:01.973410 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1aa66855-b6d1-4919-8422-65488ad85e63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:01.984094 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba4ed0-9307-48a0-9bde-913f90a49c5e service neutron] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:01.986387 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-3fba4ed0-9307-48a0-9bde-913f90a49c5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:02.010989 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba4ed0-9307-48a0-9bde-913f90a49c5e service neutron] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:02.021614 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85665d2f-ea49-49a6-a298-46452aacf9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:02.022743 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85665d2f-ea49-49a6-a298-46452aacf9a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:02.023211 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85665d2f-ea49-49a6-a298-46452aacf9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:02.023769 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85665d2f-ea49-49a6-a298-46452aacf9a0 admin admin] PATH_INFO: `/v3/projects/c3b50c56130b488e9d1d44e6af7b4ee1` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:02.026787 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85665d2f-ea49-49a6-a298-46452aacf9a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3b50c56130b488e9d1d44e6af7b4ee1)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:02.030701 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85665d2f-ea49-49a6-a298-46452aacf9a0 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:02.105416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-85665d2f-ea49-49a6-a298-46452aacf9a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3b50c56130b488e9d1d44e6af7b4ee1', 'request_id': 'req-85665d2f-ea49-49a6-a298-46452aacf9a0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:02.115372 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-85665d2f-ea49-49a6-a298-46452aacf9a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3b50c56130b488e9d1d44e6af7b4ee1', 'request_id': 'req-85665d2f-ea49-49a6-a298-46452aacf9a0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:02.115707 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa66855-b6d1-4919-8422-65488ad85e63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:02.118582 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa66855-b6d1-4919-8422-65488ad85e63 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:02.123284 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-85665d2f-ea49-49a6-a298-46452aacf9a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3b50c56130b488e9d1d44e6af7b4ee1', 'request_id': 'req-85665d2f-ea49-49a6-a298-46452aacf9a0'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:02.143608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-85665d2f-ea49-49a6-a298-46452aacf9a0 admin admin] The token cache is being invalidate because project c3b50c56130b488e9d1d44e6af7b4ee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:35:02.143848 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1436/2855] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:35:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:35:02.144331 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-85665d2f-ea49-49a6-a298-46452aacf9a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:02.145085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-85665d2f-ea49-49a6-a298-46452aacf9a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:02.146824 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1420/2856] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:35:01 2022] DELETE /identity/v3/projects/c3b50c56130b488e9d1d44e6af7b4ee1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:35:02.159000 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba5d32-ba82-471e-a1c9-6a14957b6a5c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:02.159814 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba5d32-ba82-471e-a1c9-6a14957b6a5c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:02.159814 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba5d32-ba82-471e-a1c9-6a14957b6a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:02.181551 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9bba5d32-ba82-471e-a1c9-6a14957b6a5c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:35:02.183512 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9bba5d32-ba82-471e-a1c9-6a14957b6a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:02.266834 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1437/2857] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:35:02.276488 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c03c3-f9b9-41a7-b127-2e4a2cac5a4c None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:02.277293 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c03c3-f9b9-41a7-b127-2e4a2cac5a4c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:02.277699 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c03c3-f9b9-41a7-b127-2e4a2cac5a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:02.312312 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-636c03c3-f9b9-41a7-b127-2e4a2cac5a4c None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:35:02.313770 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-636c03c3-f9b9-41a7-b127-2e4a2cac5a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:02.325752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1421/2858] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:35:02.332276 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba5d32-ba82-471e-a1c9-6a14957b6a5c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:02.351176 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba5d32-ba82-471e-a1c9-6a14957b6a5c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:02.359247 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4db21d-20db-4035-8387-d29e2be9bb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:02.359909 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4db21d-20db-4035-8387-d29e2be9bb0f admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:02.360185 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4db21d-20db-4035-8387-d29e2be9bb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:02.360457 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4db21d-20db-4035-8387-d29e2be9bb0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:02.361236 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4db21d-20db-4035-8387-d29e2be9bb0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:35:02.361610 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4db21d-20db-4035-8387-d29e2be9bb0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:02.363407 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4db21d-20db-4035-8387-d29e2be9bb0f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:02.381118 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1438/2859] 10.222.0.88 () {58 vars in 1153 bytes} [Tue Aug 23 22:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:35:02.383030 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c03c3-f9b9-41a7-b127-2e4a2cac5a4c None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:02.396199 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c03c3-f9b9-41a7-b127-2e4a2cac5a4c None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:02.404485 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c24739-068f-44bc-b181-1652027e4a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:02.405618 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c24739-068f-44bc-b181-1652027e4a50 admin admin] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:02.406128 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c24739-068f-44bc-b181-1652027e4a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:02.406615 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c24739-068f-44bc-b181-1652027e4a50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:02.407852 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c24739-068f-44bc-b181-1652027e4a50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:02.411646 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c24739-068f-44bc-b181-1652027e4a50 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:02.463548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1422/2860] 10.222.0.88 () {60 vars in 1153 bytes} [Tue Aug 23 22:35:02 2022] POST /identity/v3/projects => generated 371 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:35:02.470723 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4db21d-20db-4035-8387-d29e2be9bb0f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:02.487590 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4db21d-20db-4035-8387-d29e2be9bb0f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:02.542591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:35:02.605345 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72516a42-88bd-40f8-a2ae-a7e6a4f28a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:02.606991 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72516a42-88bd-40f8-a2ae-a7e6a4f28a0a admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:02.606991 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72516a42-88bd-40f8-a2ae-a7e6a4f28a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:02.606991 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72516a42-88bd-40f8-a2ae-a7e6a4f28a0a admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:02.607617 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72516a42-88bd-40f8-a2ae-a7e6a4f28a0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:02.610972 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72516a42-88bd-40f8-a2ae-a7e6a4f28a0a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:02.644157 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1439/2861] 10.222.0.88 () {60 vars in 1144 bytes} [Tue Aug 23 22:35:02 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:35:02.649018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c24739-068f-44bc-b181-1652027e4a50 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:02.672151 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c24739-068f-44bc-b181-1652027e4a50 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:02.681033 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ad301-3c06-486c-bd6a-03eadea40a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:02.682453 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ad301-3c06-486c-bd6a-03eadea40a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:02.682856 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ad301-3c06-486c-bd6a-03eadea40a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:02.683260 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ad301-3c06-486c-bd6a-03eadea40a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:02.684515 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ad301-3c06-486c-bd6a-03eadea40a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:02.687834 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ad301-3c06-486c-bd6a-03eadea40a00 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:02.711604 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1423/2862] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:35:02.715868 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72516a42-88bd-40f8-a2ae-a7e6a4f28a0a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:02.732877 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72516a42-88bd-40f8-a2ae-a7e6a4f28a0a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:02.739223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143f29e-c669-42db-9ea0-bc779f67e190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:02.740638 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143f29e-c669-42db-9ea0-bc779f67e190 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:02.740816 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143f29e-c669-42db-9ea0-bc779f67e190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:02.741118 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143f29e-c669-42db-9ea0-bc779f67e190 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:02.741961 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143f29e-c669-42db-9ea0-bc779f67e190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:02.743783 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143f29e-c669-42db-9ea0-bc779f67e190 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:02.769325 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1440/2863] 10.222.0.88 () {58 vars in 1122 bytes} [Tue Aug 23 22:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:35:02.774748 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ad301-3c06-486c-bd6a-03eadea40a00 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:02.785404 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ad301-3c06-486c-bd6a-03eadea40a00 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:02.790307 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd7037-c817-4555-8b7c-280359514e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:02.790999 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd7037-c817-4555-8b7c-280359514e39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:02.792554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd7037-c817-4555-8b7c-280359514e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:02.792840 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd7037-c817-4555-8b7c-280359514e39 admin admin] PATH_INFO: `/v3/projects/50a9fbbbfb5a439da09b67c2c33ce768/users/559b216124204c1fbba41b6bbf1a1c4e/roles/fad7ba55962b40459b40dcc2bad395b2` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:02.827946 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd7037-c817-4555-8b7c-280359514e39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a9fbbbfb5a439da09b67c2c33ce768, user_id=559b216124204c1fbba41b6bbf1a1c4e, role_id=fad7ba55962b40459b40dcc2bad395b2)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:02.831008 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd7037-c817-4555-8b7c-280359514e39 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:02.855416 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1424/2864] 10.222.0.88 () {60 vars in 1483 bytes} [Tue Aug 23 22:35:02 2022] PUT /identity/v3/projects/50a9fbbbfb5a439da09b67c2c33ce768/users/559b216124204c1fbba41b6bbf1a1c4e/roles/fad7ba55962b40459b40dcc2bad395b2 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:35:02.867660 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb844460-1f8e-46a2-a73b-39ed83fc4864 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:02.867938 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb844460-1f8e-46a2-a73b-39ed83fc4864 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:02.868254 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb844460-1f8e-46a2-a73b-39ed83fc4864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:02.922417 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-bb844460-1f8e-46a2-a73b-39ed83fc4864 None None] MFA Rules not processed for user `559b216124204c1fbba41b6bbf1a1c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:35:02.924933 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bb844460-1f8e-46a2-a73b-39ed83fc4864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:03.060156 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1441/2865] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:35:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:35:03.076847 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd7037-c817-4555-8b7c-280359514e39 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:03.079610 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-d6fd7037-c817-4555-8b7c-280359514e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:03.149283 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:35:03.176371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd7037-c817-4555-8b7c-280359514e39 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:03.182562 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96d28e-8018-4650-adf4-a1169dc01203 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2211e237fe554a82baf129e440a0fb43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:03.183667 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96d28e-8018-4650-adf4-a1169dc01203 service glance] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:03.184148 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96d28e-8018-4650-adf4-a1169dc01203 service glance] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:03.184894 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96d28e-8018-4650-adf4-a1169dc01203 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:03.198746 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96d28e-8018-4650-adf4-a1169dc01203 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:03.201541 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96d28e-8018-4650-adf4-a1169dc01203 service glance] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:03.213182 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1425/2866] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:35:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:35:13.056902 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:35:13.207758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=10 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:35:17.879958 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb844460-1f8e-46a2-a73b-39ed83fc4864 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:17.881557 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bb844460-1f8e-46a2-a73b-39ed83fc4864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:17.940160 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb844460-1f8e-46a2-a73b-39ed83fc4864 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:17.943759 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c100af3-f784-4a93-8ebb-fd851d601715 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:17.944356 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c100af3-f784-4a93-8ebb-fd851d601715 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:17.944635 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c100af3-f784-4a93-8ebb-fd851d601715 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:17.944905 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c100af3-f784-4a93-8ebb-fd851d601715 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:17.946687 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8c100af3-f784-4a93-8ebb-fd851d601715 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:17.956883 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c100af3-f784-4a93-8ebb-fd851d601715 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:17.958432 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c100af3-f784-4a93-8ebb-fd851d601715 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:17.967965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1442/2867] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:35:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:35:19.734759 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96d28e-8018-4650-adf4-a1169dc01203 service glance] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:19.744809 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96d28e-8018-4650-adf4-a1169dc01203 service glance] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:19.748999 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23f82d-bd0d-4855-96bb-7506e2575df0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:19.749640 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23f82d-bd0d-4855-96bb-7506e2575df0 service neutron] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:19.749916 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23f82d-bd0d-4855-96bb-7506e2575df0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:19.750187 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23f82d-bd0d-4855-96bb-7506e2575df0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:19.751981 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-1c23f82d-bd0d-4855-96bb-7506e2575df0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:19.822371 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23f82d-bd0d-4855-96bb-7506e2575df0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:19.824694 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23f82d-bd0d-4855-96bb-7506e2575df0 service neutron] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:19.835252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1426/2868] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:35:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:35:27.963429 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=26 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:35:29.830435 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=30 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:35:31.051761 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c100af3-f784-4a93-8ebb-fd851d601715 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:31.127316 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c100af3-f784-4a93-8ebb-fd851d601715 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:31.132241 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e450cf-f44f-4066-bb70-8798f98dddb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:31.132891 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e450cf-f44f-4066-bb70-8798f98dddb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:31.133207 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e450cf-f44f-4066-bb70-8798f98dddb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:31.133484 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e450cf-f44f-4066-bb70-8798f98dddb4 admin admin] PATH_INFO: `/v3/users/559b216124204c1fbba41b6bbf1a1c4e` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:31.134695 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e450cf-f44f-4066-bb70-8798f98dddb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=559b216124204c1fbba41b6bbf1a1c4e)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:31.137575 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e450cf-f44f-4066-bb70-8798f98dddb4 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:31.196427 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f3e450cf-f44f-4066-bb70-8798f98dddb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '559b216124204c1fbba41b6bbf1a1c4e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:31.212007 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f3e450cf-f44f-4066-bb70-8798f98dddb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '559b216124204c1fbba41b6bbf1a1c4e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:31.222974 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f3e450cf-f44f-4066-bb70-8798f98dddb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '559b216124204c1fbba41b6bbf1a1c4e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:31.223768 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f3e450cf-f44f-4066-bb70-8798f98dddb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '559b216124204c1fbba41b6bbf1a1c4e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:31.234039 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f3e450cf-f44f-4066-bb70-8798f98dddb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '559b216124204c1fbba41b6bbf1a1c4e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:31.241955 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1443/2869] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:35:31 2022] DELETE /identity/v3/users/559b216124204c1fbba41b6bbf1a1c4e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:35:31.246839 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75dfb6-e13c-463b-870e-247281723c83 None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:31.247415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75dfb6-e13c-463b-870e-247281723c83 None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:31.247415 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75dfb6-e13c-463b-870e-247281723c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:31.275573 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-8c75dfb6-e13c-463b-870e-247281723c83 None None] MFA Rules not processed for user `a7b11924f137447a8c88ac81ee52e9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:35:31.278172 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8c75dfb6-e13c-463b-870e-247281723c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:31.363410 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1427/2870] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:35:31.375368 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e450cf-f44f-4066-bb70-8798f98dddb4 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:31.377965 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f3e450cf-f44f-4066-bb70-8798f98dddb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:31.459164 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e450cf-f44f-4066-bb70-8798f98dddb4 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:31.463640 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525de6fc-4cc3-4477-8a0d-34a621eb3a62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:31.464275 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525de6fc-4cc3-4477-8a0d-34a621eb3a62 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:31.464563 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525de6fc-4cc3-4477-8a0d-34a621eb3a62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:31.464841 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525de6fc-4cc3-4477-8a0d-34a621eb3a62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:31.475262 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525de6fc-4cc3-4477-8a0d-34a621eb3a62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:31.476750 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525de6fc-4cc3-4477-8a0d-34a621eb3a62 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:31.489931 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1444/2871] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:35:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:35:31.883794 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75dfb6-e13c-463b-870e-247281723c83 None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:31.886807 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8c75dfb6-e13c-463b-870e-247281723c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:31.908050 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75dfb6-e13c-463b-870e-247281723c83 None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:31.916426 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60f2ac-2f76-4923-a4b2-24395835acbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:31.917785 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60f2ac-2f76-4923-a4b2-24395835acbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:31.918788 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60f2ac-2f76-4923-a4b2-24395835acbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:31.919083 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60f2ac-2f76-4923-a4b2-24395835acbe admin admin] PATH_INFO: `/v3/projects/50a9fbbbfb5a439da09b67c2c33ce768` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:31.921163 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60f2ac-2f76-4923-a4b2-24395835acbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=50a9fbbbfb5a439da09b67c2c33ce768)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:31.922913 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60f2ac-2f76-4923-a4b2-24395835acbe admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:32.004973 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-9a60f2ac-2f76-4923-a4b2-24395835acbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50a9fbbbfb5a439da09b67c2c33ce768', 'request_id': 'req-9a60f2ac-2f76-4923-a4b2-24395835acbe'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:32.014804 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-9a60f2ac-2f76-4923-a4b2-24395835acbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50a9fbbbfb5a439da09b67c2c33ce768', 'request_id': 'req-9a60f2ac-2f76-4923-a4b2-24395835acbe'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:32.022364 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-9a60f2ac-2f76-4923-a4b2-24395835acbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50a9fbbbfb5a439da09b67c2c33ce768', 'request_id': 'req-9a60f2ac-2f76-4923-a4b2-24395835acbe'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:32.041174 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-9a60f2ac-2f76-4923-a4b2-24395835acbe admin admin] The token cache is being invalidate because project 50a9fbbbfb5a439da09b67c2c33ce768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:35:32.041531 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-9a60f2ac-2f76-4923-a4b2-24395835acbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:32.042898 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-9a60f2ac-2f76-4923-a4b2-24395835acbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:35:32.045697 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1428/2872] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:35:31 2022] DELETE /identity/v3/projects/50a9fbbbfb5a439da09b67c2c33ce768 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:35:41.483566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:35:42.036391 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:35:51.112258 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525de6fc-4cc3-4477-8a0d-34a621eb3a62 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:51.114671 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-525de6fc-4cc3-4477-8a0d-34a621eb3a62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:51.234771 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525de6fc-4cc3-4477-8a0d-34a621eb3a62 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:51.344593 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac941d-2092-440c-b517-ab5fb35fe16a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d9ebd88431d46348089bc195491fca5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:51.345701 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac941d-2092-440c-b517-ab5fb35fe16a service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:51.346190 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac941d-2092-440c-b517-ab5fb35fe16a service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:51.346714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac941d-2092-440c-b517-ab5fb35fe16a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:51.349691 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-89ac941d-2092-440c-b517-ab5fb35fe16a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:51.471567 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac941d-2092-440c-b517-ab5fb35fe16a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:51.474534 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac941d-2092-440c-b517-ab5fb35fe16a service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:51.493341 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1445/2873] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:35:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:35:57.506563 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83609e4f-7fa0-4669-b5df-362cafa1023c None None] REQUEST_METHOD: `POST` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:57.507458 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83609e4f-7fa0-4669-b5df-362cafa1023c None None] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:57.507458 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83609e4f-7fa0-4669-b5df-362cafa1023c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:57.528443 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.auth.core [None req-83609e4f-7fa0-4669-b5df-362cafa1023c None None] MFA Rules not processed for user `fa949e48248747369eefd74280441b75`. Rule list: `[]` (Enabled: `True`). {{(pid=81679) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:35:57.529717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-83609e4f-7fa0-4669-b5df-362cafa1023c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:57.536768 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1429/2874] 10.222.0.88 () {58 vars in 958 bytes} [Tue Aug 23 22:35:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:35:57.923475 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac941d-2092-440c-b517-ab5fb35fe16a service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:57.924971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-89ac941d-2092-440c-b517-ab5fb35fe16a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:57.997953 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac941d-2092-440c-b517-ab5fb35fe16a service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:58.001610 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276aa8a-1c47-47f9-ab33-13e1e95b78cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:58.002218 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276aa8a-1c47-47f9-ab33-13e1e95b78cd service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:58.002501 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276aa8a-1c47-47f9-ab33-13e1e95b78cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:58.002775 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276aa8a-1c47-47f9-ab33-13e1e95b78cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:58.004566 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1276aa8a-1c47-47f9-ab33-13e1e95b78cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:58.015235 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276aa8a-1c47-47f9-ab33-13e1e95b78cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:58.016664 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276aa8a-1c47-47f9-ab33-13e1e95b78cd service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:58.025559 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1446/2875] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:35:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:35:58.482320 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83609e4f-7fa0-4669-b5df-362cafa1023c None None] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:58.483950 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-83609e4f-7fa0-4669-b5df-362cafa1023c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:58.564552 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83609e4f-7fa0-4669-b5df-362cafa1023c None None] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:58.569688 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2a576-74e1-40a2-a1cf-b2892f179796 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e642dd7a07ca4afaa40c2769299d8bad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:58.570321 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2a576-74e1-40a2-a1cf-b2892f179796 service placement] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:58.570598 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2a576-74e1-40a2-a1cf-b2892f179796 service placement] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:58.570872 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2a576-74e1-40a2-a1cf-b2892f179796 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:58.573560 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f9b2a576-74e1-40a2-a1cf-b2892f179796 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:58.586076 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2a576-74e1-40a2-a1cf-b2892f179796 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:58.587696 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2a576-74e1-40a2-a1cf-b2892f179796 service placement] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:58.599876 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1430/2876] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:35:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:36:02.568591 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276aa8a-1c47-47f9-ab33-13e1e95b78cd service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:02.571223 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1276aa8a-1c47-47f9-ab33-13e1e95b78cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:02.598153 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:36:02.718325 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276aa8a-1c47-47f9-ab33-13e1e95b78cd service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:02.730080 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ab70f-9aab-436b-b7eb-4419914eec45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:02.732338 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ab70f-9aab-436b-b7eb-4419914eec45 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:02.732835 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ab70f-9aab-436b-b7eb-4419914eec45 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:02.733343 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ab70f-9aab-436b-b7eb-4419914eec45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:02.736060 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-651ab70f-9aab-436b-b7eb-4419914eec45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:02.757504 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ab70f-9aab-436b-b7eb-4419914eec45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:02.760461 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ab70f-9aab-436b-b7eb-4419914eec45 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:02.783833 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1447/2877] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:36:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:36:08.594758 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=35 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:36:12.031202 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2a576-74e1-40a2-a1cf-b2892f179796 service placement] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:12.054461 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2a576-74e1-40a2-a1cf-b2892f179796 service placement] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:12.065661 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b2272-c8f6-43c9-b82f-b538d06da77f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:12.066683 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b2272-c8f6-43c9-b82f-b538d06da77f service nova] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:12.067113 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b2272-c8f6-43c9-b82f-b538d06da77f service nova] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:12.067369 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b2272-c8f6-43c9-b82f-b538d06da77f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:12.069947 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-963b2272-c8f6-43c9-b82f-b538d06da77f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:12.093315 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b2272-c8f6-43c9-b82f-b538d06da77f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:12.096447 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b2272-c8f6-43c9-b82f-b538d06da77f service nova] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:12.117781 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1431/2878] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:36:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:36:12.773810 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=28 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:36:16.347379 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ab70f-9aab-436b-b7eb-4419914eec45 service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:16.360894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ab70f-9aab-436b-b7eb-4419914eec45 service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:16.367255 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d5671-beb7-4359-842f-933eb7973e2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3517b3b077a4a1fa76163657b46930c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:16.367942 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d5671-beb7-4359-842f-933eb7973e2e service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:16.368514 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d5671-beb7-4359-842f-933eb7973e2e service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:16.368514 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d5671-beb7-4359-842f-933eb7973e2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:16.370861 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-445d5671-beb7-4359-842f-933eb7973e2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:16.385984 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d5671-beb7-4359-842f-933eb7973e2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:16.387648 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d5671-beb7-4359-842f-933eb7973e2e service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:16.409310 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1448/2879] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:36:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:36:22.109523 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=6 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:36:26.397227 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:37:47.466377 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b2272-c8f6-43c9-b82f-b538d06da77f service nova] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:37:47.480042 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b2272-c8f6-43c9-b82f-b538d06da77f service nova] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:37:47.487109 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ebc9b4-ea9c-4020-9d93-5c2e5dfb596c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e642dd7a07ca4afaa40c2769299d8bad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:37:47.487734 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebc9b4-ea9c-4020-9d93-5c2e5dfb596c service placement] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:37:47.488019 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebc9b4-ea9c-4020-9d93-5c2e5dfb596c service placement] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:37:47.488306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebc9b4-ea9c-4020-9d93-5c2e5dfb596c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:37:47.490905 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-f3ebc9b4-ea9c-4020-9d93-5c2e5dfb596c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:37:47.505480 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ebc9b4-ea9c-4020-9d93-5c2e5dfb596c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:37:47.507083 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ebc9b4-ea9c-4020-9d93-5c2e5dfb596c service placement] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:37:47.518736 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1432/2880] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:37:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:37:57.513636 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=6 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:37:57.520688 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d5671-beb7-4359-842f-933eb7973e2e service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:37:57.522352 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-445d5671-beb7-4359-842f-933eb7973e2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:37:57.535449 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d5671-beb7-4359-842f-933eb7973e2e service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:37:57.539531 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8a513-ce53-43f7-a781-842ebbdda720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6505100494d4496591407c48b48535ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:37:57.540135 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8a513-ce53-43f7-a781-842ebbdda720 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:37:57.540419 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8a513-ce53-43f7-a781-842ebbdda720 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:37:57.540692 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8a513-ce53-43f7-a781-842ebbdda720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:37:57.542494 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8ba8a513-ce53-43f7-a781-842ebbdda720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:37:57.611335 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8a513-ce53-43f7-a781-842ebbdda720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:37:57.612794 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8a513-ce53-43f7-a781-842ebbdda720 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:37:57.623978 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1449/2881] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:38:01.083931 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ebc9b4-ea9c-4020-9d93-5c2e5dfb596c service placement] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:38:01.099122 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ebc9b4-ea9c-4020-9d93-5c2e5dfb596c service placement] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:38:01.105036 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046a7e7-fc35-47ca-ae3f-39eb13b2b93e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e642dd7a07ca4afaa40c2769299d8bad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:38:01.106130 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046a7e7-fc35-47ca-ae3f-39eb13b2b93e service placement] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:01.106507 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046a7e7-fc35-47ca-ae3f-39eb13b2b93e service placement] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:01.106892 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046a7e7-fc35-47ca-ae3f-39eb13b2b93e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:01.109579 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-0046a7e7-fc35-47ca-ae3f-39eb13b2b93e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:38:01.124152 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046a7e7-fc35-47ca-ae3f-39eb13b2b93e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:38:01.126634 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046a7e7-fc35-47ca-ae3f-39eb13b2b93e service placement] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:38:01.149110 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1433/2882] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:38:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:38:04.440358 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8a513-ce53-43f7-a781-842ebbdda720 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:38:04.442807 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8ba8a513-ce53-43f7-a781-842ebbdda720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:38:04.468367 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8a513-ce53-43f7-a781-842ebbdda720 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:38:04.478201 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63612f7-4498-4189-b40f-bb667360072c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:38:04.479316 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63612f7-4498-4189-b40f-bb667360072c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:04.479862 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63612f7-4498-4189-b40f-bb667360072c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:04.480355 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63612f7-4498-4189-b40f-bb667360072c admin admin] PATH_INFO: `/v3/users/fa949e48248747369eefd74280441b75` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:04.482506 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63612f7-4498-4189-b40f-bb667360072c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa949e48248747369eefd74280441b75)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:38:04.486355 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63612f7-4498-4189-b40f-bb667360072c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:38:04.547323 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b63612f7-4498-4189-b40f-bb667360072c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa949e48248747369eefd74280441b75'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:04.560666 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b63612f7-4498-4189-b40f-bb667360072c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa949e48248747369eefd74280441b75'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:04.568957 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b63612f7-4498-4189-b40f-bb667360072c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa949e48248747369eefd74280441b75'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:04.569766 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b63612f7-4498-4189-b40f-bb667360072c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa949e48248747369eefd74280441b75'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:04.578036 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b63612f7-4498-4189-b40f-bb667360072c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa949e48248747369eefd74280441b75'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:04.585616 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1450/2883] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:38:04 2022] DELETE /identity/v3/users/fa949e48248747369eefd74280441b75 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:38:04.788065 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046a7e7-fc35-47ca-ae3f-39eb13b2b93e service placement] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:38:04.806836 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046a7e7-fc35-47ca-ae3f-39eb13b2b93e service placement] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:38:04.867320 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca92619-5d9f-4562-9547-ae84e126661f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:38:04.867957 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca92619-5d9f-4562-9547-ae84e126661f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:04.868277 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca92619-5d9f-4562-9547-ae84e126661f admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:04.868548 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca92619-5d9f-4562-9547-ae84e126661f admin admin] PATH_INFO: `/v3/projects/77cc91a5c0bc4e5888460d7618594895` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:04.869705 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca92619-5d9f-4562-9547-ae84e126661f admin admin] RBAC: Authorizing `identity:delete_project(project_id=77cc91a5c0bc4e5888460d7618594895)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:38:04.871616 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca92619-5d9f-4562-9547-ae84e126661f admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:38:04.923755 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-dca92619-5d9f-4562-9547-ae84e126661f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77cc91a5c0bc4e5888460d7618594895', 'request_id': 'req-dca92619-5d9f-4562-9547-ae84e126661f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:04.931559 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-dca92619-5d9f-4562-9547-ae84e126661f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77cc91a5c0bc4e5888460d7618594895', 'request_id': 'req-dca92619-5d9f-4562-9547-ae84e126661f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:04.937288 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-dca92619-5d9f-4562-9547-ae84e126661f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77cc91a5c0bc4e5888460d7618594895', 'request_id': 'req-dca92619-5d9f-4562-9547-ae84e126661f'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:04.951108 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-dca92619-5d9f-4562-9547-ae84e126661f admin admin] The token cache is being invalidate because project 77cc91a5c0bc4e5888460d7618594895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:38:04.951399 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-dca92619-5d9f-4562-9547-ae84e126661f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:04.952021 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-dca92619-5d9f-4562-9547-ae84e126661f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:04.953607 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1434/2884] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:38:04 2022] DELETE /identity/v3/projects/77cc91a5c0bc4e5888460d7618594895 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:38:04.957860 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63612f7-4498-4189-b40f-bb667360072c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:38:04.959398 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b63612f7-4498-4189-b40f-bb667360072c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:38:04.974058 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63612f7-4498-4189-b40f-bb667360072c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:38:05.032046 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6661e66-5a95-408d-9881-19be26b633d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:38:05.032735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6661e66-5a95-408d-9881-19be26b633d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:05.032999 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6661e66-5a95-408d-9881-19be26b633d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:05.033406 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6661e66-5a95-408d-9881-19be26b633d8 admin admin] PATH_INFO: `/v3/users/016968b282b74ecc837fc8756d4be9ee` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:05.034492 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6661e66-5a95-408d-9881-19be26b633d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=016968b282b74ecc837fc8756d4be9ee)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:38:05.036425 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6661e66-5a95-408d-9881-19be26b633d8 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:38:05.041981 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:38:05.089320 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b6661e66-5a95-408d-9881-19be26b633d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '016968b282b74ecc837fc8756d4be9ee'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:05.101573 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b6661e66-5a95-408d-9881-19be26b633d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '016968b282b74ecc837fc8756d4be9ee'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:05.110366 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b6661e66-5a95-408d-9881-19be26b633d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '016968b282b74ecc837fc8756d4be9ee'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:05.111057 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b6661e66-5a95-408d-9881-19be26b633d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '016968b282b74ecc837fc8756d4be9ee'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:05.119076 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b6661e66-5a95-408d-9881-19be26b633d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '016968b282b74ecc837fc8756d4be9ee'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:05.127242 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1451/2885] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:38:04 2022] DELETE /identity/v3/users/016968b282b74ecc837fc8756d4be9ee => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:38:05.362751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca92619-5d9f-4562-9547-ae84e126661f admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:38:05.364190 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-dca92619-5d9f-4562-9547-ae84e126661f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:38:05.381814 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca92619-5d9f-4562-9547-ae84e126661f admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:38:05.444865 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:38:05.445529 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:05.445811 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:05.446085 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee admin admin] PATH_INFO: `/v3/projects/898f0f02fb794dcc96209124944cfe31` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:05.447215 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=898f0f02fb794dcc96209124944cfe31)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:38:05.449146 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:38:05.470910 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:38:05.497877 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '898f0f02fb794dcc96209124944cfe31', 'request_id': 'req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:05.504457 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '898f0f02fb794dcc96209124944cfe31', 'request_id': 'req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:05.510554 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '898f0f02fb794dcc96209124944cfe31', 'request_id': 'req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:05.526100 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee admin admin] The token cache is being invalidate because project 898f0f02fb794dcc96209124944cfe31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81679) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:38:05.526461 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:05.527053 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:38:05.528608 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1435/2886] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:38:05 2022] DELETE /identity/v3/projects/898f0f02fb794dcc96209124944cfe31 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:38:15.122513 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:38:15.522853 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:39:07.047151 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6661e66-5a95-408d-9881-19be26b633d8 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:39:07.048714 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b6661e66-5a95-408d-9881-19be26b633d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:39:07.134465 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:39:07.136079 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:39:07.156215 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6661e66-5a95-408d-9881-19be26b633d8 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:39:07.223668 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fbaeb6-8fc1-4142-b61e-5140d2fc57ee admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:39:07.229442 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64044256-91b9-4e7c-9bd7-0a8760c5f734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:39:07.230094 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64044256-91b9-4e7c-9bd7-0a8760c5f734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:39:07.230642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64044256-91b9-4e7c-9bd7-0a8760c5f734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:39:07.230642 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64044256-91b9-4e7c-9bd7-0a8760c5f734 admin admin] PATH_INFO: `/v3/users/7a43d4ec9b1346e3a69b01d3d6e83a98` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:39:07.231773 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64044256-91b9-4e7c-9bd7-0a8760c5f734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a43d4ec9b1346e3a69b01d3d6e83a98)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:39:07.233869 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64044256-91b9-4e7c-9bd7-0a8760c5f734 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:39:07.241859 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87fe43-a03d-463c-a1e4-1236812203e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e642dd7a07ca4afaa40c2769299d8bad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:39:07.242800 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87fe43-a03d-463c-a1e4-1236812203e2 service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:39:07.243292 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87fe43-a03d-463c-a1e4-1236812203e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:39:07.243735 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87fe43-a03d-463c-a1e4-1236812203e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:39:07.246485 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9a87fe43-a03d-463c-a1e4-1236812203e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:39:07.265033 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87fe43-a03d-463c-a1e4-1236812203e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:39:07.268487 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87fe43-a03d-463c-a1e4-1236812203e2 service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:39:07.286515 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1452/2887] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:39:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:39:07.293306 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-64044256-91b9-4e7c-9bd7-0a8760c5f734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a43d4ec9b1346e3a69b01d3d6e83a98'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:07.294210 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-64044256-91b9-4e7c-9bd7-0a8760c5f734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a43d4ec9b1346e3a69b01d3d6e83a98'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:07.306535 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-64044256-91b9-4e7c-9bd7-0a8760c5f734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a43d4ec9b1346e3a69b01d3d6e83a98'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:07.320917 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-64044256-91b9-4e7c-9bd7-0a8760c5f734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a43d4ec9b1346e3a69b01d3d6e83a98'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:07.335918 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-64044256-91b9-4e7c-9bd7-0a8760c5f734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a43d4ec9b1346e3a69b01d3d6e83a98'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:07.343863 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1436/2888] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:39:07 2022] DELETE /identity/v3/users/7a43d4ec9b1346e3a69b01d3d6e83a98 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:39:07.529595 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87fe43-a03d-463c-a1e4-1236812203e2 service placement] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:39:07.540728 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87fe43-a03d-463c-a1e4-1236812203e2 service placement] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:39:07.606468 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:39:07.607098 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:39:07.607374 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:39:07.607644 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb admin admin] PATH_INFO: `/v3/projects/94a56fcfc827430fb1b73dcfd186024d` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:39:07.608763 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=94a56fcfc827430fb1b73dcfd186024d)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:39:07.610717 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:39:07.650181 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:39:07.662696 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94a56fcfc827430fb1b73dcfd186024d', 'request_id': 'req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:07.669470 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94a56fcfc827430fb1b73dcfd186024d', 'request_id': 'req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:07.677990 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94a56fcfc827430fb1b73dcfd186024d', 'request_id': 'req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:07.690014 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb admin admin] The token cache is being invalidate because project 94a56fcfc827430fb1b73dcfd186024d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:39:07.690304 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:07.691008 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:07.693305 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1453/2889] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:39:07 2022] DELETE /identity/v3/projects/94a56fcfc827430fb1b73dcfd186024d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:39:07.697567 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64044256-91b9-4e7c-9bd7-0a8760c5f734 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:39:07.699029 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-64044256-91b9-4e7c-9bd7-0a8760c5f734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:39:07.712073 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64044256-91b9-4e7c-9bd7-0a8760c5f734 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:39:07.774763 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30334a-3df5-4cf6-aff7-5096c009a9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:39:07.775414 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30334a-3df5-4cf6-aff7-5096c009a9f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:39:07.775718 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30334a-3df5-4cf6-aff7-5096c009a9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:39:07.776018 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30334a-3df5-4cf6-aff7-5096c009a9f3 admin admin] PATH_INFO: `/v3/users/91d9c01bf013482ba5cda775bb2972e6` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:39:07.777212 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30334a-3df5-4cf6-aff7-5096c009a9f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91d9c01bf013482ba5cda775bb2972e6)` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:39:07.779155 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30334a-3df5-4cf6-aff7-5096c009a9f3 admin admin] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:39:07.806546 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:39:07.833520 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8d30334a-3df5-4cf6-aff7-5096c009a9f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91d9c01bf013482ba5cda775bb2972e6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:07.834086 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8d30334a-3df5-4cf6-aff7-5096c009a9f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91d9c01bf013482ba5cda775bb2972e6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:07.841195 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8d30334a-3df5-4cf6-aff7-5096c009a9f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91d9c01bf013482ba5cda775bb2972e6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:07.853161 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8d30334a-3df5-4cf6-aff7-5096c009a9f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91d9c01bf013482ba5cda775bb2972e6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:07.861121 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.notifications [None req-8d30334a-3df5-4cf6-aff7-5096c009a9f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91d9c01bf013482ba5cda775bb2972e6'} {{(pid=81679) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:07.867752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1437/2890] 10.222.0.88 () {58 vars in 1224 bytes} [Tue Aug 23 22:39:07 2022] DELETE /identity/v3/users/91d9c01bf013482ba5cda775bb2972e6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:39:08.045894 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:39:08.047384 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:39:08.062963 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd42c4-7821-4263-ba6e-0bd4891ed8eb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:39:08.125419 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e796c3-7f70-4786-8838-f1aa5f15754f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b11924f137447a8c88ac81ee52e9ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe428a23adc74bbab88ff935243cc4e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:39:08.125971 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e796c3-7f70-4786-8838-f1aa5f15754f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:39:08.126255 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e796c3-7f70-4786-8838-f1aa5f15754f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:39:08.126532 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e796c3-7f70-4786-8838-f1aa5f15754f admin admin] PATH_INFO: `/v3/projects/c582df208dec464aa84d4cf33190607a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:39:08.127669 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e796c3-7f70-4786-8838-f1aa5f15754f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c582df208dec464aa84d4cf33190607a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:39:08.130007 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e796c3-7f70-4786-8838-f1aa5f15754f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:39:08.179650 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-93e796c3-7f70-4786-8838-f1aa5f15754f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c582df208dec464aa84d4cf33190607a', 'request_id': 'req-93e796c3-7f70-4786-8838-f1aa5f15754f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:08.185463 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-93e796c3-7f70-4786-8838-f1aa5f15754f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c582df208dec464aa84d4cf33190607a', 'request_id': 'req-93e796c3-7f70-4786-8838-f1aa5f15754f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:08.193371 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-93e796c3-7f70-4786-8838-f1aa5f15754f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c582df208dec464aa84d4cf33190607a', 'request_id': 'req-93e796c3-7f70-4786-8838-f1aa5f15754f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:08.204581 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-93e796c3-7f70-4786-8838-f1aa5f15754f admin admin] The token cache is being invalidate because project c582df208dec464aa84d4cf33190607a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:39:08.205693 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-93e796c3-7f70-4786-8838-f1aa5f15754f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:08.206428 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-93e796c3-7f70-4786-8838-f1aa5f15754f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:39:08.208382 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1454/2891] 10.222.0.88 () {58 vars in 1233 bytes} [Tue Aug 23 22:39:08 2022] DELETE /identity/v3/projects/c582df208dec464aa84d4cf33190607a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:39:17.863587 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:39:18.201589 dv-cinder-850321-4-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:DELETE=12,keystone:SELECT=62,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:43:35.226665 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30334a-3df5-4cf6-aff7-5096c009a9f3 admin admin] Authenticating user token {{(pid=81679) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:43:35.228318 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-8d30334a-3df5-4cf6-aff7-5096c009a9f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:43:35.371717 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30334a-3df5-4cf6-aff7-5096c009a9f3 admin admin] Validating token access rules against request {{(pid=81679) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:43:35.433855 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c3170-0d1a-47a2-80b8-665cdcd8b42f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e642dd7a07ca4afaa40c2769299d8bad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a7317af325f485e8fae49631cb5bc20', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81679) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:43:35.434471 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c3170-0d1a-47a2-80b8-665cdcd8b42f service placement] REQUEST_METHOD: `GET` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:43:35.434752 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c3170-0d1a-47a2-80b8-665cdcd8b42f service placement] SCRIPT_NAME: `/identity` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:43:35.435025 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c3170-0d1a-47a2-80b8-665cdcd8b42f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81679) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:43:35.436751 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.fernet_utils [None req-865c3170-0d1a-47a2-80b8-665cdcd8b42f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81679) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:43:35.449252 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c3170-0d1a-47a2-80b8-665cdcd8b42f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:43:35.450871 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c3170-0d1a-47a2-80b8-665cdcd8b42f service placement] RBAC: Authorization granted {{(pid=81679) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:43:35.461333 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: [pid: 81679|app: 0|req: 1438/2892] 10.222.0.88 () {60 vars in 1365 bytes} [Tue Aug 23 22:43:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:43:45.456593 dv-cinder-850321-4-iscsi devstack@keystone.service[81679]: DEBUG dbcounter [-] [81679] Writing DB stats keystone:SELECT=89 {{(pid=81679) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}